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Abstract 


Recently,  the  Air  Force  Civil  Engineer  Center  (AFCEC)  identified  that  the  Mission 
Dependency  Index  (MDI)  had  issues  with  reflecting  the  criticality  of  some  mission  sets.  The 
MDI  is  a  constructed  value  assigned  to  assets  that  reflects  the  consequence  of  failure.  The 
primary  mission  sets  having  MDI  issues  were  non-flightline  assets.  The  current  Air  Force  MDI 
metric  relies  on  data  collected  using  the  Naval  Facilities  Engineering  Command  (NAVFAC) 
methodology  and  adapts  the  data  by  using  facility  categorization  codes.  The  result  is  a  method 
that  compares  alternatives  to  each  other  to  develop  an  individual  asset’s  MDI  value.  As  a 
corrective  measure  to  this  methodology,  non-flightline  centric  mission  sets  have  been  allowed  to 
adjust  (i.e.  increase)  their  asset  MDI  values.  This  modification  in  MDI  values  has  led  to  inflation 
of  the  metric. 

To  address  the  issue,  this  research  focuses  on  how  the  MDI  values  should  be  assigned  by 
examining  both  public  and  private  methodologies.  Leveraging  the  Delphi  technique  and  Value 
Focused  Thinking  (VFT),  three  models  are  created  to  suggest  the  proper  inputs  that  should  be 
considered  when  producing  the  MDI  values  for  the  Air  Force’s  assets.  The  models  inputs  were 
interruptability,  redundancy,  replaceability  and  the  number  of  missions  affected.  The  Delphi 
panel  weighted  each  input,  and  the  resulting  VFT  models  displayed  the  mirco  (local-level) 
perspective,  macro  (headquarters-level)  perspective,  and  the  combined  perspective. 
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A  DELPHI  STUDY  USING  VALUE-FOCUSED  THINKING  FOR  UNITED  STATES 


AIR  FORCE  MISSION  DEPENDENCY  INDEX  VALUES 

I.  Introduction 

Determining  how  to  prioritize  funding  actions  is  a  challenge  for  any  corporation.  The 
process  of  prioritizing  projects  is  no  different  for  Air  Force  Civil  Engineers.  The  Air  Force  is 
responsible  for  an  infrastructure  portfolio  of  139,556  assets  worth  $263.43  billion  (Sitzabee  & 
Harnly,  2013).  Placing  the  value  of  these  assets  in  perspective,  the  gross  domestic  product  of  the 
country  of  Finland  in  2013  was  $259  billion  (CIA,  2014).  The  number  of  assets  the  Air  Force  is 
responsible  for  is  around  three  times  the  amount  the  Target  Corporation  is  responsible  for  and  is 
comprised  of  615  million  square  feet  of  building  space  (Byers,  2012).  To  demonstrate  the 
magnitude  of  building  space,  the  office  space  on  Manhattan  is  520  million  square  feet  (Rudder 
Property  Group,  2015). 

Properly  maintaining  this  large  asset  portfolio  is  a  challenge  for  the  Civil  Engineer  career 
field.  This  challenge  is  a  result  of  applying  funding  to  prioritized  infrastructure  maintenance  and 
repair  projects,  and  the  interdependencies  that  occur  from  supporting  the  various  mission  sets  in 
the  enterprise  portfolio.  Fiscally,  in  order  to  support  the  portfolio,  “the  Air  Force  allocates  $2.5 
billion  annually  to  maintenance  and  repair  projects”  (Sitzabee  &  Harnly,  2013:56),  which 
equates  to  almost  1  percent  of  the  overall  portfoilio  value  each  year.  “An  appropriate  budget 
allocation  for  routine  M&R  for  substanstail  inventory  of  facilities  will  typically  be  in  the  range 
of  2  to  4  percent”  (National  Academy  Press,  1990:xii).  With  the  Air  Force  only  receiving  a 
fraction  of  the  suggested  funding  to  maintain  a  large  real  property  portfolio,  each  funding 
decision  must  be  made  to  maximize  the  cost  efficiency. 
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The  Air  Force’s  facilities  are  interdependent  to  maximize  efficiency  to  support  various 


missions.  Rinaldi  et  al.(  2001),  demonstrated  that  evaluating  all  of  the  interdependencies  of 


infrastructure  can  be  difficult,  as  shown  by  Figure  1 .  With  all  of  the  various  dimensions  of 


independencies,  it  is  difficult  to  take  into  account  every  dimension  in  the  prioritization  of  funds. 


Figure  1.  Dimensions  of  Infrastructure  Interdependencies  (Rinaldi  et  al.,  2001) 


In  recent  years,  there  have  been  three  prioritization  models  applied  to  the  Air  Force 


infrastructure  portfolio.  The  goal  of  these  models  was  to  decide  where  to  best  allocate  the  funds 


and  to  take  into  account  many  of  the  dimensions  of  infrastructure  interdependencies.  The  two 


most  recent  models  employed  the  mission  dependency  index  (MDI)  as  an  input.  “Mission 
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Dependency  Index  is  the  value  an  asset  brings  to  the  perfonnance  of  the  mission  as  detennined 
by  the  governing  agency”  (Federal  Real  Property  Council,  2011: 12).  The  MDI  is  a  government 
tenn  for  an  Asset  Prioritization  Index  (API).  Like  MDI,  an  API  measures  the  consequence  of 
failure  associated  with  a  single  building  or  asset.  This  research  examines  the  MDI  metric  and 
suggests  areas  for  improvement.  Chapter  I  provides  a  brief  background  of  the  MDI,  the  problem 
statement,  the  research  objective,  investigative  questions,  the  research  approach,  assumptions, 
and  limitations  of  this  research  effort,  and  concludes  with  an  overview  of  the  remaining  chapters. 

Mission  Dependency  Index  Background 

Executive  Order  (EO)  13327,  Federal  Real  Property  Asset  Management,  was  published 
in  2004  in  an  effort  to  mandate  a  more  efficient  way  to  manage  federal  real  property  assets 
through  the  implementation  of  asset  management.  Specifically,  E.O.  13327  charges  federal 
agencies  to  create  “life-cycle  cost  estimations  associated  with  the  agency’s  prioritized  actions” 
(Executive  Order  13327,  2004:5898).  Prior  to  the  MDI,  the  Air  Force  implemented  the  Facility 
Investment  Metric  (FIM).  The  FIM  model  split  all  assets  into  facility  classes  based  on  the 
category  code  (CATCODE)  of  the  individual  asset.  Eleven  different  types  of  facility  classes 
existed  ranging  from  operations  and  training  to  community  support  to  delineate  the  mission  set 
an  asset  supported.  After  detennining  the  appropriate  facility  class,  the  user  would  be  required 
to  rank  the  facility’s  criticality.  Air  Force  Instruction  (AFI)  32-1032,  Planning  and  Programming 
Appropriated  Funded  Maintenance,  Repair,  and  Construction  Projects  (2003),  defined  the  impact 
ratings  to  address  how  the  failure  of  the  facility  would  result  in  the  failure  of  a  mission.  The 
three  impact  ratings  were  defined  as  critical,  degraded,  and  essential  to  reflect  the  mission  effect 
of  an  asset  failing.  To  place  a  project  in  an  impact  category,  it  must  have  met  the  requirements 
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outlined  in  AFI  32-1032.  After  bases  and  Major  Commands  (MAJCOMs)  submitted  the 
requirements  to  Headquarters  Air  Force  (HAF),  the  HAF  staff  consolidated  the  requirements. 
After  consolidation,  HAF  validated  each  requirement  was  placed  in  the  proper  facility  class  and 
impact  rating  category.  Following  the  validation,  the  projects  were  prioritized  for  funding.  This 
process  met  the  desired  effect  of  the  executive  order;  however,  the  timeline  and  person-hours 
associated  with  this  methodology  did  not  allow  for  quick  prioritization  of  assets.  Figure  2  from 
AFI  32-1032  (2003)  shows  the  FIM  requirements  matrix. 


IMPACT  RATINGS 

Facility  Class 

Critical 

Degraded 

Essential 

Operations  and  Training 

Mobility' 

Maintenance  and  Production 

RDT&E 

Supply 

Medical 

Administrative 

Community  Support 

MFH 

Dormitories 

Utilities  and  Ground  Improvements 

Figure  2.  FIM  Requirements  Matrix  (AFI  32-1032,  2003) 
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In  2008,  the  Air  Force  Civil  Engineer  (CE)  career  field  identified  the  need  to  create  a 
system  where  all  Air  Force  assets  can  be  prioritized  using  a  common  metric.  The  common 
metric  was  intended  to  allow  for  a  more  transparent  and  effective  funding  model  as  part  of  the 
new  system.  One  of  the  models  the  Air  Force  considered  but  did  not  adopt  included  the  risk 
management  methodology  that  combined  the  probability  and  severity  of  an  asset’s  failure.  The 
two  metrics  applied  to  the  suggested  model  were  the  facility  condition  index  (FCI)  and  MDI. 
These  two  new  metrics  allowed  leadership  to  compare  assets  portfolio  wide  in  a  standardized 
fashion.  Madaus  (2008)  created  Figure  3  to  help  display  these  categories  to  the  decision-makers 
to  aid  in  the  selection  of  the  best  course  of  action  (COA). 
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Figure  3.  MDI  vs.  FCI  Decision  Matrix  (Madaus,  2008) 
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The  other  model  was  a  balanced  scorecard  that  incorporated  many  of  the  strategic  goals 
of  the  CE  career  field.  The  categories  on  the  scorecard  addressed  health  and  safety  compliance, 
the  facility  condition,  MDI,  local  mission  impact,  cost  efficiency,  and  the  MAJCOM  priority. 
Each  category  received  a  weight  to  contribute  to  the  overall  score.  Once  the  overall  score  was 
calculated  for  each  requirement,  all  projects  were  prioritized  and  the  infrastructure  funding  was 
distributed  accordingly.  As  shown  in  Figure  4,  the  balanced  scorecard  model  could  address 
multiple  corporate  goals;  therefore,  it  was  selected  and  implemented  through  2013  (Headquarters 
Air  Force,  2010). 
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Figure  4.  The  Infrastructure  Prioritization  Balanced  Scorecard  (Headquarters  Air  Force,  2010) 
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When  the  balanced  scorecard  method  was  implemented,  it  used  MDI  as  one  of  the  inputs. 
The  Air  Force  had  to  create  a  methodology  to  produce  an  interim  MDI  for  facilities  to  use  in 
prioritization  method.  The  interim  method  relied  on  MDI  values  produced  from  the  Naval 
Facilities  Engineering  Command  (NAVFAC)  model  and  assigned  each  identified  MDI  to  each 
respective  CATCODE.  The  data  set  was  assigned  by  CATCODE  similar  to  the  National  Park 
Service  (NPS)  methodology.  The  NPS  methodology  assigns  the  MDI  values  based  on  the 
CATCODE  rather  than  individual  assets;  the  NPS  uses  this  approach  because  it  does  not  require 
the  same  amount  of  data  collection  as  the  NAVFAC  model.  The  team  producing  the  interim 
MDI  merged  the  two  methodologies  because  there  was  “no  clear  Industry  or  OSD  standard 
method  to  calculate  MDI”  (Madaus,  2008:6).  This  method  was  implemented  with  the  intent  to 
revisit  the  methodology  and  create  a  more  accurate  and  repeatable  metric. 

After  publishing  the  new  MDI  values  for  CATCODEs,  the  MAJCOMs  identified 
numerous  MDI-to-CATCODE  mismatches  that  were  not  fulfilling  the  intent  of  measuring 
criticality  and  replaceability.  MAJCOMs  having  a  primary  mission  other  than  flight  operations 
had  the  largest  issue  with  the  method.  For  example,  Air  Education  and  Training  Command’s 
(AETC)  mission  is  to  “Recruit,  train  and  educate  Ainnen  to  deliver  airpower  for  America” 
(AETC,  2014:1).  Because  of  AETC’s  educational  focus,  their  mission  critical  facilities  are 
classrooms,  dormitories,  and  training  facilities.  As  a  result,  each  MAJCOM  could  submit 
recommended  adjustments  to  a  specific  CATCODE’s  MDI  value.  Initially,  when  an  adjustment 
occurred  to  bring  an  MDI  value  up,  the  overall  distribution  of  the  CATCODEs  was  renormalized 
to  maintain  HAF’s  desired  nonnal  distribution  and  their  desired  range  of  values  from  1-100. 

Over  time,  that  corrective  action  no  longer  occurred.  As  each  justification  was  approved,  the 
MDI  distribution  shifted,  becoming  left  skewed,  and  compacting  the  effective  range  to  32-99. 
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Problem  Statement 


Currently,  the  MDI  values  have  a  skewed  distribution  to  the  left  and  the  effective  range  is 
less  than  was  originally  intended.  Originally,  the  MDI  was  meant  to  be  a  normalized  curve  with 
a  mean  value  of  50.  In  Figure  5,  the  numbers  of  occurrences  that  specific  MDI  values  are 
assigned  to  a  CATCODE  are  plotted  against  the  MDI  values  in  bins  of  five  (Avery,  2013). 


Figure  5.  USAF  CATCODE  Distribution  vs.  MDI  Values  (Avery,  2013) 


Air  Force  Civil  Engineer  Center  (AFCEC)  recognized  there  was  an  issue  with  the  MDI 
values  in  the  fall  of  2013  (Avery,  2013).  In  response  to  the  issue,  AFCEC  created  a  working 
group  to  produce  a  rule  set  for  the  career  field.  The  rule  set  would  give  the  proper  guidance  to 
correct  an  incorrect  MDI  at  all  levels.  This  effort  would  assist  in  identifying  improper  MDI 
values  for  specific  buildings  and  ensured  the  identified  facilities  receive  funding.  Although  this 
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addresses  the  issue  of  individual  MDI  values,  there  is  still  a  concern  that  inflation  may  continue 
without  a  method  to  govern  the  distribution  of  the  MDI  values.  If  the  current  inflation  continues, 
the  effective  range  of  the  metric  decreases,  thereby  reducing  the  decision-making  value. 


Research  Objective  and  Investigative  Questions 

Given  the  problem  stated  above,  this  research  focuses  on  answering  the  question,  “How 
can  AFCEC  create  a  process  to  prevent  MDI  value  inflation  while  maintaining  a  usable  metric 
for  funding  decisions?”  To  help  address  the  research  question,  four  investigative  questions  were 
developed  to  help  provide  a  complete  answer.  These  investigative  questions  were: 

1 .  What  are  the  inputs  needed  to  define  the  MDI  value  of  a  building? 

2.  What  weight  should  be  applied  to  each  of  the  inputs  when  assigning  an  MDI  value  to 
a  facility  or  asset? 

3.  What  is  the  desired  distribution  of  the  MDI  values? 

4.  According  to  CE  SMEs,  how  should  the  MDI  be  applied  to  the  prioritization  of 
infrastructure  funding  decisions? 

The  questions  mentioned  above  were  developed  in  an  effort  to  better  understand  the  MDI  tool,  as 
well  as  how  the  metrics  produced  by  the  MDI  system  should  be  applied  in  the  CE  career  field. 
The  current  MDI  does  not  have  a  methodology  other  than  comparing  a  specific  asset’s  MDI  to 
another  asset’s  MDI.  Keeney  (1996)  warns  against  this  approach  by  stating,  “focusing  on 
alternatives  is  a  limited  way  to  think  through  decision  situations.  It  is  reactive,  not  proactive.” 
(Keeney,  1996:537).  As  a  solution  to  the  alternative-focused  thought  process,  Keeney  suggests 
that  decision-makers  should  focus  on  desired  values  when  making  decisions.  Once  these  values 
have  been  identified,  the  interactions  between  the  values  should  also  be  identified  to  better 
understand  the  opportunity  accompanying  the  decision.  In  establishing  a  metric,  determining  the 
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proper  usage  is  vital  to  creating  the  decision-making  system.  After  clearly  identifying  the 
definition  and  proper  usage  of  the  MDI  metric  system,  the  research  can  provide  AFCEC  with  a 
metric  to  implement  funding  decisions  relating  to  the  Air  Force  infrastructure  portfolio. 

Methodology 

To  answer  the  questions  above,  the  research  employed  two  methodologies:  a  Delphi 
study  and  Value  Focused  Thinking  (VFT).  The  Delphi  method  was  performed  with  the  help  of 
CE’s  Senior  Leaders.  The  Delphi  method  takes  advantage  of  subject  matter  experts’  (SME) 
knowledge  and  experience  in  an  iterative  process  to  provide  the  research  with  a  well-informed 
decision  towards  policymaking.  Throughout  the  Delphi  method,  a  VFT  method  was  applied  to 
help  develop  a  framework  to  assist  in  producing  the  MDI  values  aligned  with  the  Air  Force’s 
goals  rather  than  comparing  each  asset  to  all  other  assets  to  create  a  value. 

Assumptions/Limitations 

The  business  rules  published  on  29  January  2014  by  AFCEC  established  a  new 
prioritization  model  (AFCEC,  2014).  The  MDI  metric  is  one  of  the  inputs  for  the  prioritization 
model.  The  application  of  MDI  in  the  model  is  the  main  driving  factor  in  this  research  effort.  If 
a  new  prioritization  model  is  developed,  the  MDI  metric  will  need  to  be  revisited  to  ensure  it 
reflects  the  desired  information.  The  research  assumed  all  facility  conditions  and  commander’s 
preferences  are  accounted  for  by  the  other  metrics  applied  to  the  funding  model.  The  research 
goal  was  to  correct  the  overall  MDI  distribution  problems,  so  it  was  not  be  able  to  address  all 
individual  infrastructure  scenarios. 
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Overview 


This  chapter  addressed  the  challenges  associated  with  infrastructure  funding  and  more 
specifically  the  challenge  the  Air  Force  faces  with  the  current  MDI  metric.  This  thesis  follows  a 
five-chapter  format.  Chapter  II  summarizes  the  literature  reviewed  throughout  the  research. 
Chapter  III  addresses  the  Delphi  and  VFT  methodologies  used  to  suggest  the  new  MDI  model. 
Chapter  IV  looks  at  the  data  acquired  through  the  Delphi  methodology  and  how  the  data  was 
applied  to  create  a  VFT  model  in  addition  to  answering  the  research  questions.  Finally,  Chapter 
V  summarizes  the  results  of  the  research  and  suggests  additional  research  streams. 
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II.  Literature  Review 


This  chapter  provides  background  material  related  to  accomplishing,  creating,  and  using 
an  Asset  Prioritization  Index  (API).  The  first  part  of  the  chapter  focuses  on  the  application  of  the 
API  approach  to  federal  assets  and  the  creation  of  the  Mission  Dependency  Index  (MDI)  metric. 
A  background  on  the  Naval  Facilities  Engineering  Command  (NAVFAC)  MDI  model  is  given  to 
show  a  specific  federal  application.  The  second  section  addresses  examples  of  how  the  private 
industry  has  approached  creating  an  API.  Specifically,  the  private  industry  methods  examined 
include:  risk  based  investment  trade-off,  American  Society  of  Testing  Materials  (ASTM),  and 
Massachusetts  Institute  of  Technology  (MIT).  The  chapter  concludes  with  a  summary  of  the 
methods  reviewed. 

Federal  MDI  Approach 

The  requirement  for  an  MDI  value  application  to  federal  infrastructure  portfolios 
originated  from  Executive  Order  13327  which  mandated  that  federal  “agencies  shall  recognize 
the  importance  of  real  property  resources  through  increased  management  attention,  the 
establishment  of  clear  goals  and  objectives,  improved  policies  and  levels  of  accountability” 
(Executive  Order  13327,  2004:5897).  To  fulfill  the  requirements  of  the  EO,  federal  agencies  had 
to  establish  asset  management  (AM)  practices.  Specifically,  EO  13327  charges  organizations  to 
take  actions  to  prioritize  assets  to  “improve  the  operational  and  financial  management  of  the 
agency’s  real  property  inventory”  (Executive  Order  13327,  2004:5898).  When  prioritizing 
actions,  the  executive  branch  adopted  the  risk  management  approach  of  measuring  probability 
and  consequence.  The  International  Infrastructure  Management  Manual  reinforces  this  approach 
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stating,  “A  combination  of  likelihood  and  consequence  of  failure  can  provide  an  overall  measure 
of  the  level  of  risk”  (New  Zealand  Asset  Management  Support,  2011).  The  executive  branch 
adopted  the  concept  of  an  API  and  created  the  term  MDI  to  attempt  to  assign  each  asset  to  the 
mission  it  supports.  Specifically,  the  MDI  was  meant  to  identify  the  consequence  of  an 
individual  asset  failing.  In  the  National  Academy  Press  report  titled,  “Committing  to  the  Cost  of 
Ownership,”  it  is  made  clear  that  unlike  private  businesses,  when  a  defense  asset  fails,  the 
potential  consequences  include  the  same  risks  the  private  sector  encounters,  in  addition  to  a  loss 
in  readiness  and  a  “domino  effect”  due  to  the  interconnectivity  of  the  military  systems  (National 
Academy  Press,  1990).  These  defense  unique  consequences  could  lead  to  breaches  in  national 
defense  (National  Academy  Press,  1990).  The  literature  reveals  different  organizations  of  the 
government  initially  approached  assigning  consequences  to  resources  differently  (National  Park 
Service,  2013;  Antehnan,  Dempsey,  &  Brodt,  2008;  Madaus,  2008).  Over  time,  the  NAVFAC 
method  was  overwhelmingly  adopted  by  the  Navy,  Coast  Guard,  Anny,  and  National 
Aeronautics  and  Space  Administration  (Grussing  et  al.,  2010).  The  NAVFAC  approach  is 
examined  in  the  following  section. 


NAVFAC  MDI  Model 

The  model  developed  by  the  NAVFAC  measures  the  consequences  of  facility  failure  by 
examining  interruptability,  relocateability,  and  replaceability  (Antelman,  Dempsey,  &  Brodt, 
2008).  The  NAVFAC  model  uses  a  structured  survey  (see  Appendix  A)  given  to  commanders  to 
establish  how  important  each  of  their  assets  is  to  their  mission.  It  also  determines  how  important 
other  commanders’  assets  are  to  their  mission.  Using  the  structured  survey,  the  MDI  team  would 
apply  the  values  created  from  the  responses  to  the  matrix  in  Figure  6  and  detennine  the  score  for 
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mission  dependency  within  an  organization  (MDwithm)-  Similarly,  the  MDI  team  would  apply  the 
responses  acquired  about  other  mission  sets  to  the  matrix  in  Figure  7  to  determine  the  score 
associated  with  mission  dependency  between  organizations  (MDBetween). 
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After  collecting  the  values  from  the  matrices  above,  the  NAVFAC  team  averaged  the 
responses  from  the  various  agencies,  counted  how  many  agencies  consider  each  asset  to  be 
mission  critical  ( n ),  and  then  placed  all  of  the  variables  into  an  equation  to  produce  the  proper 
MDI  value.  The  formulation  shown  in  Equation  1  is  used  to  calculate  the  MDI  for  each  asset. 
The  values  for  MDwuhin  and  MDBetween  are  the  values  retrieved  from  the  matrices  in  Figure  6  and 
Figure  7,  respectively.  The  n  is  evaluated  using  a  natural  log  function  to  reflect  the  law  of 
diminishing  returns.  As  the  equation  suggests,  the  structured  survey  must  be  accomplished  for 
each  asset.  Figure  8  displays  the  weight  associated  with  the  three  inputs  and  brings  attention  to 
the  local  level,  which  has  the  most  input  toward  the  asset’s  MDI  value. 


MDI  =  26.54  +  0  +  0. 1  Ln  („)]  -  25.54  ( 1) 


■  MD  Within 

MDBetween 

■  n 


Figure  8.  NAVFAC  Model  Variables  Weights  toward  MDI  value 
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This  approach  requires  a  survey  team  approximately  one  week  and  $40,000  to  $75,000 
per  site  to  accomplish  the  required  data  acquisition  to  make  the  model  accurate  (Grussing  et  ah, 
2010).  Although  this  model  requires  continual  financial  resources  and  man-hours  to  maintain, 
“the  MDI  has  been  recognized  by  the  US  General  Services  Administration  in  2003  as  a  ‘Best 
Practice’”  (Antehnan  et  ah,  2008).  As  a  result,  the  United  States  Coast  Guard  (USCG),  National 
Aeronautical  and  Space  Administration  (NASA),  and  the  United  States  Army  (USA)  have 
adopted  this  model  (Grussing  et  al.,  2010). 

Although  this  model  produces  a  usable  product,  it  has  four  logistical  challenges 
associated  with  it.  The  first  challenge  is  the  availability  of  a  database  powerful  enough  to  store 
and  produce  the  required  queries  given  the  large  amount  of  data  required  to  provide  an  accurate 
input.  The  United  States  Navy  has  111  bases,  and  each  base  must  input  the  dependence  variable 
of  all  the  other  base’s  facilities  (Department  of  Defense,  2013).  Because  of  this,  there  can  be  up 
to  1 1 1  inputs  per  asset  for  the  MDbetween  value.  If  the  Air  Force  adapted  this  method,  there  would 
be  up  to  185  inputs  for  each  asset  (Department  of  Defense,  2013).  To  assemble  this  amount  of 
information,  the  database  for  the  Air  Force  would  need  to  be  around  three  times  bigger  than 
NAVFAC’s  database  because  of  the  additional  infonnation  associated  with  identifying  all  of  the 
relationships. 

The  second  logistical  challenge  focuses  on  cost.  When  leadership,  missions,  or 
operations  tempo  change,  the  structured  survey  must  be  re-accomplished.  As  Grussing  et  al. 
(2010)  estimates,  the  cost  associated  with  this  method  is  $40,000  to  $75,000  per  base.  An 
optimistic  estimation  of  the  initial  expenditure  for  the  Air  Force,  assuming  that  leadership  stays 
in  place  for  3  years  and  the  surveys  cost  $40,000  would,  be  $7.4M.  To  account  for  leadership, 
mission,  and  operations  tempo  changes,  an  annual  fee  for  this  type  of  system  could  be  upwards 
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of  $2.5M.  The  annual  fee  accounts  for  the  assumption  that  one  third  of  the  leadership  changes 
every  year,  thus  driving  the  need  for  the  survey  to  be  re-accomplished.  For  a  pessimistic 
estimate,  assumptions  were  that  leadership  changed  every  2  years  and  a  survey  cost  of  $75,000, 
thus  leading  to  an  initial  expense  of  $  13.9M  with  an  annual  fee  of  $6.9M.  If  the  annual  amounts 
of  funds  are  not  maintained  in  the  budget,  the  MDI  values  may  not  be  maintained  properly. 

The  third  logistical  challenge  involves  the  lack  of  proper  CATCODE  assignment.  This 
concern  only  affects  the  model  when  it  is  applied  at  the  CATCODE  level  instead  of  the  asset- 
specific  level.  For  example,  CATCODE  442421  is  a  ‘controlled  humidity  warehouse.’  This 
assignment  accurately  identifies  the  majority  of  assets  in  that  category;  however,  it  does  not 
identify  a  facility  that  may  support  a  unique  mission  set  such  as  special  operations  or  cyber 
warfare  assets.  The  unique  mission  sets  may  require  the  MDI  value  to  be  greater  than  59  to 
reflect  the  relationship  to  the  mission  properly.  Other  CATCODES  exist  for  warehouses  that 
have  higher  MDI  values  to  more  accurately  reflect  the  specific  asset.  These  corrections  will  need 
to  take  place  at  the  base  level  by  the  Real  Property  Office  to  ensure  proper  identification  of  each 
facility. 

The  last  logistical  challenge  accompanying  this  model  is  educating  leadership  to  answer 
the  survey  consistently.  To  ensure  consistency,  the  NAVFAC  uses  two  teams  to  minimize  the 
variability  among  survey  results.  The  Air  Force  would  need  to  determine  a  procedure  to  ensure 
the  model  was  applied  consistently  at  all  installations.  This  procedure  could  be  achieved  by  a 
specific  set  of  business  rules,  or  a  traveling  survey  team  much  like  the  NAVFAC.  Each  of  the 
logistical  challenges  presented  need  to  be  addressed  for  this  model  to  function  as  intended  for  the 
Air  Force. 
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Private  Industry  Asset  Prioritization  Index  Applications 

Three  private  methods  were  researched  to  see  if  they  could  be  adopted  by  the  Air  Force. 
The  first  method,  risk-based  investment  trade-off  approach,  examines  a  situation  where  the  asset 
is  functional  and  compares  it  to  a  situation  when  the  asset  is  broken.  The  difference  in  capacity 
is  the  value  used  to  prioritize  facilities.  The  second  method  requires  pair-wise  comparisons  of  all 
the  alternatives  to  detennine  the  overall  and  relative  importance  values.  The  last  method 
involves  creating  a  model  regardless  of  inputs  and  then  applying  the  validated  model  to 
decisions.  The  following  sections  will  address  each  method  in  further  detail. 

Risk-Based  Investment  Trade-Off  Approach 

Taillandier,  Sauce,  and  Bonetto  (2009)  followed  a  different  approach  to  quantifying 
consequences  for  decision-making.  Their  research  observes  the  mission  impact  of  an  unintended 
event  (UE)  occurring  and  the  difference  in  potential  damage.  Two  scenarios  with  input  from  the 
tactical  and  managerial  level  were  examined  to  measure  the  potential  damages.  In  one  scenario, 
an  event  occurred  and  in  the  other  scenario,  no  event  occurred.  The  difference  in  potential 
damage  displayed  the  amount  of  consequences  associated  with  an  asset.  When  comparing  all 
assets’  potential  damage  differences,  the  decision-makers  could  readily  identify  the  most 
important  asset.  This  methodology  is  cumbersome,  but  it  reflects  the  views  from  multiple  levels 
of  managers  associated  with  the  asset.  Figure  9  shows  the  model  to  help  predict  the  consequence 
amount. 
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Figure  9.  Decision  Model  to  Produce  a  Consequence  Domain 
(Taillandier,  Sauce,  &  Bonetto,  2009:789) 


Patterson  and  Apostolakis  (2007)  applied  a  similar  methodology  in  an  attempt  to  identify 
critical  locations  across  multiple  infrastructures.  The  authors  created  a  metric,  the  Bimbaum 
importance  measure  (IM),  represented  by  Equation  2.  “The  Birnbaum  IM  describes  the  change 
in  risk  to  user  j  for  infrastructure  k  when  element  y  switches  from  available  to  unavailable” 
(Patterson  &  Apostolakis,  2007: 1 189). 


(i) 


The  if  variable  represents  the  number  of  failed  elements  contributing  to  a  system’s  failure,  and 
the  U  variable  represents  the  elements  still  in  working  condition.  This  method  shows  that  if 
certain  assets  in  the  system  malfunction,  there  is  a  cascading  impact  on  the  user.  After  the 
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Bimbaum  IM  is  assigned  to  each  building  in  the  system,  the  critical  nodes  in  the  infrastructure 
can  be  identified.  While  the  way  it  assigns  the  IM  to  the  buildings  is  similar  to  Taillandier  et  al. 
(2009),  this  method  finds  the  consequence  of  failure  of  all  infrastructure  sections  in  a  portfolio. 

The  risk-based  investment  trade-off  approach  could  be  applied  to  Air  Force  facilities,  but 
three  challenges  would  have  to  be  overcome.  The  first  challenge  would  be  to  establish  a  rule  set 
for  consistency  when  evaluating  different  scenarios.  Detennining  how  many  levels  of  asset 
managers  to  involve  in  determining  the  consequence  index  would  be  the  second  challenge. 

Asset  managers  at  each  level  of  an  organization  have  different  agendas  and  assigned  missions, 
which  affects  how  he  or  she  views  the  importance  of  an  individual  asset.  The  final  challenge 
would  involve  identifying  a  metric  to  help  analyze  all  of  the  consequence  indexes.  As  the 
authors  point  out,  “Converting  every  data  into  a  monetary  equivalent  data  system  does  not  make 
it  possible  for  decision-makers  to  put  the  different  kinds  of  consequences  (human,  commercial, 
material,  etc.)  in  perspective”  (Taillandier,  Sauce,  &  Bonetto,  2009:789).  If  these  challenges  are 
addressed  correctly,  this  methodology  could  be  applied  to  the  Air  Force’s  asset  portfolio. 

American  Society  of  Testing  Materials  (ASTM)  Model 

The  ASTM  model  presented  in  standards  El 765-1 1,  Standard  Practice  for  Applying 
Analytical  Hierarchy  Process  (AHP)  to  Multi-attribute  Decision  Analysis  of  Investments  Related 
to  Buildings  and  Build  Systems,  and  E2495-13,  Standard  Practice  for  Prioritizing  Asset 
Resources  in  Acquisition,  Utilization,  and  Disposition,  provides  a  systematic  approach  on  how  to 
produce  a  multi-attribute  decision  analysis  tool.  “The  practice  presents  a  procedure  for 
calculating  and  interpreting  AHP  scores  of  a  project’s  total  overall  desirability”  (  ASTM, 

2011: 1).  The  standard  demonstrates  how  to  produce  a  value  tree  to  make  decisions.  In  addition 
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to  the  value  tree  solution,  it  displays  an  example  of  how  to  perform  AHP  on  a  “finite  and 
generally  small  set  of  discrete. . .  options”  (ASTM,  2011 :2). 

As  shown  in  Figure  10,  to  perfonn  an  AHP  analysis,  all  assets  must  be  listed  in  a  matrix 
comparing  one  asset  to  all  others  (i.e.  pair-wise  comparison).  When  the  weights  are  assigned, 
the  relative  importance  is  given  to  each  pairing  using  a  scale.  After  all  of  the  paired  comparisons 
among  alternatives  are  complete,  the  weights  of  each  alternative  are  created  by  solving  for  the 
eigenvector,  represented  as  e*  in  Equation  3.  M  is  the  matrix  of  paired  comparisons  and  Xmax 
represents  the  principal  eigenvalue  of  the  matrix  M.  Once  e*  is  found,  its  value  represents  the 
relative  weight  of  that  item  compared  to  all  other  items  appearing  in  the  analysis.  If  all  of  the  e* 
values  are  sorted,  a  prioritized  list  will  be  created. 

^maXe*=Me*  (2) 

To  properly  apply  the  AHP  process  to  the  facilities  in  the  Air  Force,  a  matrix  would  need 
to  be  created  with  n(n- 1)/2  elements,  where  n  represents  each  facility.  If  the  AHP  method  was 
applied  using  all  139,556  facilities,  it  would  require  over  9  billion  paired  comparisons  to 
accomplish.  If  the  analysis  was  accomplished  at  the  CATCODE  level,  where  there  are  966 
categories,  it  would  require  only  466,095  paired  comparisons.  Although  requiring  fewer 
comparisons  overall,  choosing  the  CATCODE  method  would  still  require  a  large  amount  of  data 
collection  and  accompanying  decisions  made  at  the  appropriate  level.  The  final  way  to 
appropriately  use  the  AHP  would  be  to  use  it  within  each  MDI  value.  This  application  would 
help  establish  a  relative  mission  importance  within  each  MDI  ranking.  The  Air  Force  could  use 
this  methodology  if  it  was  limited  to  a  level  that  would  result  in  a  manageable  amount  of  paired 
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comparisons.  The  level  that  produced  the  manageable  amount  of  comparisons  needs  to  be 
determined  by  the  leadership  responsible  for  allocating  resources.  If  a  level  is  selected  without 
the  resources  to  support  it,  this  method  will  be  unattainable.  Overall,  the  AHP  method  could 
prove  to  be  the  most  accurate  method  of  modeling  all  assets  against  each  other  to  produce  a 
prioritized  list,  but  it  would  require  a  tremendous  amount  of  participation  from  leadership  to 
achieve  the  desired  outcome. 
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(ASTM,  2011) 
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Massachusetts  Institute  of  Technology  (MIT)  Model 

To  ensure  the  infrastructure  support  budget  is  applied  to  the  correct  assets  responsibly, 
MIT  developed  a  model  to  “apply  limited  resources  to  the  most  important  needs  first,  support 
consistent,  repeatable,  and  defendable  prioritization  decisions,  consider  the  impact  of  risk  on 
these  decisions,  be  flexible  and  easy  to  use,  and  enable  careful  and  thoughtful  consideration  of 
alternative  choices  while  supporting  consensus”  (Karydas  &  Gifun,  2006:85).  The  process  MIT 
developed  combined  three  different  methodologies.  The  entire  process  “developed  originally  at 
MIT  for  the  prioritization  of  safety  and  operational  experience  in  nuclear  power  plants  was  [then] 
adopted  and  modified  by  facilities  for  its  use”  (Karydas  &  Gifun,  2006:85).  The  prioritization 
process  consists  of  six  steps:  develop  the  project  selection  process,  define  impact  categories  and 
perfonnance  measures,  weight  impact  categories  and  perfonnance  measures,  define  and  weight 
constructed  scales,  check  for  consistency,  and  check  for  validity  and  reliability. 

The  first  step  is  to  develop  the  project  selection  process.  In  this  step,  MIT’s  decision¬ 
makers  placed  the  project  in  one  of  the  four  categories  shown  in  Figure  1 1  (Karydas  &  Gifun, 
2006).  When  the  board  of  decision-makers  is  presented  with  a  project,  it  will  be  prioritized  only 
if  it  has  a  moderate  to  high  cost  and  does  not  need  to  be  accomplished  immediately.  The  projects 
that  must  be  accomplished  immediately  for  safety  reasons  or  other  factors  are  placed  in  the 
“Must  Do”  category.  On  the  other  end  of  the  spectrum,  if  a  project  does  not  add  any  value  to  the 
mission  of  the  campus,  it  is  placed  in  the  “Must  Not  Do”  category.  If  the  project  is  small  or  the 
request  is  accomplishing  preventive  maintenance,  the  staff  will  accomplish  it  as  labor  becomes 
available. 
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Figure  1 1 .  MIT  Prioritization  Process  Map 


After  the  project  selection  process  is  defined,  the  decision-makers  develop  impact 
categories  (IC)  and  perfonnance  measures  (PM).  In  MIT’s  case,  the  board  created  six  impact 
categories  and  ten  performance  measures.  Each  category  and  measure  is  defined  specifically  to 
help  the  members  of  the  board  understand  the  true  intention  of  the  model.  The  definitions  of  the 
categories  for  MIT  are  shown  Appendix  B.  After  all  desired  perfonnance  measures  are 
identified,  a  weighting  scheme  is  applied  using  an  analytical  hierarchy  process  (AHP).  The 
results  of  the  MIT  AHP  are  shown  in  Figure  12.  Each  value  shown  is  out  of  a  total  possible 
value  of  1.0  to  show  the  importance  of  each  category  in  comparison  with  other  categories. 
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Figure  12.  MIT  Prioritization  Value  Tree 
(Karydas  &  Gifun,  2006:88) 


Following  the  weighting  of  all  the  performance  measures,  the  board  then  defines  and 
weights  the  constructed  scales  within  each  performance  measure  individually.  The  authors  used 
three  levels  of  values  for  each  perfonnance  measure.  The  scale  included  specific  descriptions  to 
ensure  reliability.  Each  level  on  the  scale  had  an  individual  weight  developed  from  a  value 
function.  The  value  function  is  an  equation  developed  from  the  input  of  experts  to  assign  various 
weights  to  the  levels  within  a  measure  (Kirkwood,  1997).  In  MIT’s  application,  disutility  curves 
were  used  as  the  value  functions  to  “depict  Facilities  preference  for  risk  adversity”  (Karydas  & 
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Gifun,  2006).  The  disutility  curves  were  developed  by  the  respective  subject  matter  experts  and 
then  verified  by  the  board  prior  to  being  incorporated  into  the  overall  model.  An  example  of  the 
different  levels  and  the  disutility  curve  is  shown  in  Figure  13. 
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Figure  13.  Example  of  MIT  Constructed  Scale  and  Disutility  Curve 
(Karydas  &  Gifun,  2006:92) 


The  authors  identified  the  final  steps  of  this  methodology  as  checking  for  consistency, 
validity,  and  reliability.  MIT  accomplished  the  final  steps  by  leveraging  the  Expert  Choice® 
application  and  benchmarking.  After  all  of  the  steps  are  accomplished,  the  model  can  be  applied 
with  the  use  of  decision-makers.  Karydas  and  Gifun  (2006)  stated  that  even  after  the  model  had 
been  applied  to  all  projects  to  provide  prioritization,  the  board  could  revisit  the  list  to  provide 
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one  final  check  on  the  prioritization  to  ensure  each  project  was  ranked  as  desired.  This  model 
applies  a  defined  scale  to  each  performance  measure  to  better  prioritize  a  portfolio  of  assets.  If 
an  organization  systematically  defines  the  impact  categories,  perfonnance  measures,  and  value 
functions,  this  model  could  be  used  as  a  template  for  implementation. 

The  MIT  model  methodology  has  been  applied  to  a  number  of  systems  to  accurately 
capture  the  risk  associated  with  an  asset.  Specifically,  Koonce,  Apostolakis,  and  Cook  (2008) 
applied  the  model  to  the  North  American  Electrical  Reliability  Council’s  bulk  power  grid  from 
the  point  of  view  of  the  electric  supplier.  Their  analysis  provided  a  “systematic  process  that 
produces  a  ranking  of  elements  within  the  bulk  power  grid  for  random  failures  and  malevolent 
acts”  (Koonce,  Apostolakis,  &  Cook,  2008: 182).  The  advantage  of  focusing  on  the  values 
associated  with  an  asset  and  creating  a  model  from  those  values  is  that  the  method  assists  the 
model  creator  in  eliminating  bias  toward  a  specific  asset  and  encourages  portfolio  optimization. 

With  some  modification,  the  model  could  be  used  as  a  starting  point  to  provide  a  better 
funding  decision  model  for  Air  Force  projects.  Specifically,  it  could  be  applied  to  develop  a  set 
of  perfonnance  measures  relating  to  the  MDI.  After  each  performance  measure  is  established,  a 
value  function  can  be  produced  to  display  the  differences  among  similar  assets  that  otherwise 
would  be  rated  similarly.  While  the  MIT  model  helps  standardize  responses,  there  are  three 
issues  with  its  implementation  in  the  Air  Force.  The  first  issue  is  detennining  the  appropriate 
level  for  the  final  deliberations  to  occur  in  the  Air  Force.  When  the  inputs  are  placed  into  the 
model,  the  board  at  MIT  verifies  a  rough  draft  of  the  prioritization  list.  This  methodology  may 
have  scaling  issues  when  trying  to  adapt  it  to  the  Air  Force.  If  the  prioritization  list  were  verified 
and  presumably  changed  at  the  wing,  MAJCOM,  and  HAF  levels,  the  original  intent  of  the  order 
of  the  projects  may  change  completely  during  the  bureaucratic  process.  The  second  issue 
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involves  accessibility  to  the  project  champion  or  the  subject  matter  experts  (SME)  throughout  the 
deliberations  stages.  At  the  lower  levels,  the  access  will  not  be  an  issue;  however,  at  the  strategic 
levels,  the  staff  will  often  not  have  the  appropriate  time  to  talk  to  every  SME  about  each  project. 
The  final  implementation  issue  is  to  define  adequately  each  performance  measure  and  level  on 
the  constructed  scale.  If  the  definitions  are  not  distinguishable  among  various  levels,  the 
interpretation  by  the  user  could  threaten  the  validity  of  the  entire  prioritization  model.  To 
successfully  adopt  this  model,  each  of  these  issues  would  need  to  be  addressed. 

Summary  of  Private  Industry  Asset  Prioritization  Index  Applications 

All  three  methods  presented  have  worked  in  specific  applications  of  the  private  sector. 
The  common  theme  when  adapting  the  methods  to  the  Air  Force  portfolio  is  scaling.  All 
methods  require  data  collection,  analysis,  and  storage  that  will  add  cost  to  the  present 
methodology.  Despite  these  challenges,  all  of  the  methods  would  provide  a  more  defendable 
process  in  creating  asset-specific  MDI  values. 

Chapter  Summary 

In  this  literature  review,  four  different  approaches  to  developing  an  MDI  metric  were 
examined.  Each  method  had  pros  and  cons  and  required  differing  amounts  of  labor  and 
resources  to  establish  and  maintain.  The  common  theme  developed  from  the  models  was  the 
importance  of  knowing  the  capability  the  asset  provides  to  the  mission.  Once  each  asset  is  tied 
to  a  function,  the  importance  of  it  not  being  there  could  be  addressed  quickly  and  the  effects  of  it 
not  functioning  would  be  clear.  In  the  next  chapter,  the  methodology  for  how  the  research 
creates  inputs  for  the  MDI  metric  is  discussed. 
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III.  Methodology 


After  studying  government  and  private  industry  methods  to  establish  an  asset 
prioritization  index  (API),  the  Massachusetts  Institute  of  Technology  (MIT)  model  was  adopted 
for  this  research.  This  methodology  was  selected  because  of  the  consistent  process  created  while 
still  allowing  the  leadership  to  revisit  the  prioritization  before  the  distribution  of  funding.  The 
MIT  approach  involves  a  group  of  stakeholders  indentifying  criteria  to  establish  a  Value  Focused 
Thinking  (VFT)  model.  The  desired  effect  of  this  research  was  to  provide  a  method  that,  when 
all  assets  have  been  assigned  an  MDI  value,  will  result  in  the  desired,  nonnal  distribution  with  no 
inflation.  As  stated  before,  the  current  MDI  system  operates  in  the  range  of  32-99  because  of 
inflation.  The  goal  of  this  research  was  to  find  an  appropriate  method  that  returns  MDI  values 
ranging  from  1-100.  The  variability  and  effective  range  are  important  because  these 
characteristics  inform  the  decision-makers  about  the  differences  between  assets.  When  an  index 
has  a  smaller  range,  the  diversity  of  the  portfolio  is  not  demonstrated  when  compared  to  the 
larger  range.  Guided  by  the  desired  outputs  of  the  process,  a  Delphi  study  was  performed  to 
identify  the  inputs  for  the  MDI  metric.  The  Delphi  questionnaire’s  data  was  employed  with  the 
Value  Focused  Thinking  (VFT)  method  to  create  the  process  of  determining  an  asset’s  MDI 
value. 

Value  Focused  Thinking  Method 

A  VFT  study  is  a  methodology  used  for  gathering  data  about  inputs  of  a  process  to 
generate  a  model  that  ties  strategic  goals  to  routine  operations  (Keeney,  1992).  VFT  pre- 
establishes  the  different  input  values  applied  in  organizational  decision-making  instead  of  relying 
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on  alternative  focused  thinking  (AFT),  the  process  of  selecting  the  strongest  alternative.  VFT 
involves  “a  shift  to  this  way  of  thinking  about  decisions. . .  because  values  guide  not  only  the 
creation  of  better  alternatives  but  the  identification  of  better  decision  situations”  (Keeney, 
1996:538).  Figure  14  demonstrates  all  of  the  characteristics  of  VFT. 


Figure  14.  Overview  of  Value  Focused  Thinking 
(Keeney,  1992:24) 


Leon  (1999)  perfonned  studies  to  further  outline  the  benefits  of  VFT  (1999).  These 
benefits  included  the  notion  that,  alternatives  with  more  innovative  characteristics  are  included, 
the  range  of  alternatives  included  becomes  wider,  the  future  consequences  of  decisions  are  taken 
more  into  account,  alternatives  that  at  first  glance  would  not  be  considered  are  integrated,  and 
consequences  that  are  more  desirable  are  considered.  Once  produced,  a  VFT  model  allows 
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decisions  to  be  made  based  on  established  criteria  rather  than  using  a  comparison  of  ah 
alternatives  in  an  individual  subjective  manner  (Keeney,  1992). 

The  VFT  process  also  has  a  number  of  challenges  identified  by  Keeney  (1996).  The  first 
challenge  is  the  initial  identification  of  fundamental  objectives  for  the  organization.  The  team 
producing  the  VFT  model  needs  to  ensure  the  objectives  identified  tie  to  the  company’s  goals 
rather  than  a  specific  alternative.  The  second  challenge  is  the  initial  time  required  to  produce  the 
VFT  model.  Keeney  emphasizes  that  if  these  challenges  are  overcome,  the  VFT  model  will 
produce  superior  results  when  compared  to  the  AFT. 

Delphi  Method 

In  1999,  Rowe  and  Wright  (1999:354)  characterized  the  three  main  features  of  a  Delphi 
study  as  “anonymity,  iteration,  [and]  controlled  feedback.”  These  features  are  achieved  through 
multiple  rounds  of  questions  in  which  expert  opinions  are  gathered  and  discussed  within  the 
group  in  an  effort  to  gamer  consensus  and/or  highlight  differences  in  opinions  on  the  topic  of 
discussion.  Throughout  the  iterative  process,  inputs  are  detennined  to  identify  and  maximize 
decision  opportunities.  Figure  15  shows  the  typical  flow  of  the  steps  in  a  Delphi  process. 
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Figure  15.  Steps  of  a  Delphi  Process 
(adapted  from  Skulmoski  et  al.,  2007:3) 


The  Delphi  rounds  produced  inputs  for  the  MDI  values,  as  well  as  helped  develop  an 
importance  associated  with  each  input.  The  Delphi  collected  questionnaire  data  from  Air  Force 
CE  senior  leaders  and  SMEs.  The  first  round  was  focused  on  identifying  issues  with  the  current 
MDI  system  as  observed  by  the  panel  members.  After  the  issues  were  identified,  the  second 
round  was  focused  on  identifying  possible  inputs  that  should  be  considered  when  creating  the 
MDI  metric.  The  third  round  attempted  to  bring  the  panel  members  to  a  consensus  on  the  values 
required  to  produce  a  usable  MDI  value.  A  copy  of  the  study’s  overview  that  was  provided  to 
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the  Delphi  participants  is  in  Appendix  C  as  well  as  the  letter  from  AFCEC  in  Appendix  J.  After 
all  rounds  were  complete,  the  data  gathered  from  the  Delphi  was  used  to  create  and  produce  a 
VFT  model. 


Delphi  Panel  Composition 

This  study  selected  CE  Career  field  leaders  and  subject  matter  experts  (SME)  as  panel 
members  because  they  are  the  experts  on  how  to  manage  the  USAF  infrastructure.  To  qualify  as 
a  senior  leader,  the  participants  needed  to  be  a  senior  Field  Grade  Officer  or  equivalent  to 
represent  the  strategic  level  of  decision-makers,  have  a  minimum  of  15  years  experience 
managing  infrastructure  assets,  and  have  a  working  knowledge  of  the  CE  prioritization  model. 

To  be  considered  a  subject  matter  expert,  the  panel  member  must  have  a  minimum  of  3  years 
experience  prioritizing  infrastructure  asset  projects  using  MDI  as  one  of  the  inputs,  working 
knowledge  of  the  CE  prioritization  model,  and  specific  knowledge  of  CE  funding  and 
prioritization  actions  in  support  of  public  infrastructure.  The  SMEs  only  required  3  years  of 
experience  with  MDI  because  MDI  has  only  been  applicable  in  the  Air  Force  prioritization 
efforts  for  6  years.  AFCEC  provided  a  group  of  individuals  that  met  these  requirements. 

The  secondary  reason  the  panel  members  were  selected  was  these  individuals  were  a 
sample  of  convenience.  They  were  identified  by  AFCEC  as  possible  participants  because  of  ease 
of  access  (they  were  part  of  the  organization),  and  their  knowledge  on  the  subject  matter.  This 
sample  provided  the  data  necessary  to  conduct  the  Delphi  study.  The  Delphi  questionnaire’s 
data  retrieved  then  supported  the  development  of  the  VFT  hierarchy.  The  inputs  for  the  first 
round  of  the  Delphi  were  created  from  the  literature  review;  subsequent  rounds  then  adapted 
questions  and  issues  as  the  participants  identified  concerns.  The  goal  of  the  study  was  to  receive 
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feedback  from  each  mission  set  (MAJCOMs)  and  the  overall  corporate  goals  (AFCEC).  The 
feedback  from  different  backgrounds  assisted  in  producing  the  inputs  for  the  MDI  metric  that 
reflect  the  importance  of  an  infrastructure  asset  in  a  repeatable  manner. 

Analysis  of  VFT  Application 

After  the  inputs  were  collected  through  the  Delphi,  the  inputs  were  used  to  create  the 
impact  categories  (IC)  or  values  of  MDI.  The  ICs  are  the  characteristics  that  should  be 
addressed  when  creating  the  MDI  of  the  asset.  After  each  IC  was  identified,  the  perfonnance 
measures  (PM)  were  determined.  The  PMs  address  how  each  IC  is  measured.  The  ICs  and  PMs 
were  placed  into  a  hierarchy  as  shown  in  Figure  12.  “Desirable  properties  for  a  value  hierarchy 
include  completeness,  nonredundancy,  decomposability,  operability,  and  small  size”  (Kirkwood, 
1997:16).  The  completeness  characteristic  is  when  all  layers  of  the  hierarchy  can  represent  every 
aspect  of  the  desired  decision  (Kirkwood,  1997).  Nonredundancy  must  be  met  in  a  hierarchy  to 
ensure  there  is  no  overlapping  between  the  tiers  (Kirkwood,  1997).  In  other  words,  this 
requirement  states  that  all  inputs  in  a  tier  must  be  mutually  exclusive  of  each  other. 
Decomposability  can  also  be  described  as  independence.  When  a  value  is  being  determined  for 
one  objective  in  the  hierarchy,  the  resultant  value  cannot  affect  another  objectives  value.  The 
operability  and  small  size  characteristics  ensure  the  hierarchy  can  be  easily  used  by  the  desired 
audience  (Kirkwood,  1997).  To  achieve  all  of  these  properties,  the  first  five  steps  of  Shovaik’s 
(2001)  ten-step  VFT  process  shown  in  Table  1  were  used.  The  scope  of  this  research  was 
focused  solely  on  providing  a  repeatable  methodology  to  develop  the  Air  Force  MDI  values.  It 
did  not  address  the  implementation  of  the  model  and  therefore  only  addressed  the  first  five  steps 
of  the  VFT  process. 
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Table  1.  Ten  Step  VFT  Process  (Shoviak,  2001) 


Step 

Description 

1 

Problem  Identification 

2 

Create  Value  Hierarchy 

3 

Develop  Evaluation  Measures 

4 

Create  Value  Functions 

5 

Weight  Value  Hierarchy 

6 

Alternative  Generation 

7 

Alternative  Scoring 

8 

Deterministic  Analysis 

9 

Sensitivity  Analysis 

10 

Conclusions  and  Recommendations 

The  VFT  method  allows  there  to  be  unique  distributions  and  weights  applied  to  reflect 
the  overall  desires  of  the  leadership.  Because  of  this  characteristic,  the  MDI  values  produced  can 
be  adjusted  to  follow  the  desired  range  and  distribution  identified  by  the  Delphi  panel.  In 
addition,  the  MDI  metric  is  meant  to  measure  criticality  in  a  similar  value  across  the  asset 
portfolio.  Depending  on  priorities,  some  mission  sets  or  locations  may  be  more  critical. 


Threats  to  Validity  and  Reliability 

The  methodology  presented  relies  heavily  on  the  Delphi  panel  balancing  the  corporate 
goals  with  the  mission-specific  goals.  Because  the  population  size  is  small,  the  validity  of  the 
weights  is  threatened  if  the  panel  cannot  reach  a  consensus.  The  reliability  is  threatened  if  the 
PMs  are  not  defined  properly.  If  the  different  levels  of  the  PMs  do  not  have  distinct  definitions, 
each  user  of  the  model  could  produce  a  separate  MDI  value  for  the  same  asset  based  on  the 
user’s  interpretation  of  the  definitions.  If  the  model  yields  different  results  for  the  same  asset 
from  run  to  run,  the  PMs  will  need  to  be  made  clearer  to  avoid  the  ambiguity  causing  the 
inconsistency. 


35 


Chapter  Summary 

The  research  used  two  different  methodologies  at  different  stages  to  determine  the  best 
MDI  model  for  the  USAF.  The  methodologies  were  a  combination  of  qualitative  and 
quantitative  approaches  in  an  effort  to  capture  the  art  and  science  aspects  of  asset  management. 
The  resulting  VFT  model,  developed  with  feedback  from  the  Delphi  panel,  should  have  the 
capability  to  adapt  to  changes  in  the  leadership  prioritization  while  providing  a  standardized 
process  to  produce  the  MDI  metric. 
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IV.  Analysis  and  Results 


This  chapter  addresses  the  results  and  themes  from  each  round  of  the  Delphi  approach,  as 
well  as  the  resulting  value  focused  thinking  (VFT)  model  that  was  developed.  Each  round  of 
Delphi  questions  are  in  Appendixes  D,  F,  and  H,  respectively.  The  specific  coding  of  the 
answers  are  in  Appendixes  E,  G  and  I.  The  panel  members  specified  by  Air  Force  Civil 
Engineer  Center  (AFCEC)  were  identified  to  ensure  feedback  received  was  from  qualified 
individuals  who  understand  multiple  missions  throughout  the  Air  Force  and  how  the 
infrastructure  supports  varying  mission  sets.  The  panel  members  who  participated  in  all  three 
rounds  had  a  combined  experience  of  over  150  years  supporting  infrastructure  for  the  public  and 
private  sector.  These  panel  members’  individual  experiences  varied  from  base  level  to 
Headquarters  Air  Force  (HAF).  The  mission  sets  supported  by  the  members  included:  Air  Force 
Global  Strike  Command,  Air  Force  Space  Command,  National  Guard  Bureau,  Air  Education 
Command,  Air  Combat  Command,  Air  Force  Special  Operations  Command,  Pacific  Air  Forces, 
Air  Force  Materiel  Command,  United  States  Air  Forces  in  Europe,  Tactical  Air  Command,  and 
Strategic  Air  Command.  The  last  two  commands  in  the  list  of  mission  sets  represented  no  longer 
exist;  however,  they  are  mentioned  to  display  the  wide  experience  of  the  panel  members. 
Throughout  the  rounds,  the  panel  focused  on  VFT  rather  than  alternative-focused  thinking  (AFT) 
to  help  produce  inputs  that  could  apply  to  any  mission  set  rather  than  specific  mission  sets.  After 
each  round,  the  responses  were  coded  and  graduate  students  at  the  Air  Force  Institute  of 
Technology  verified  10%  of  the  coding  as  suggested  by  Lombard  et  al.  (2002). 
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Delphi  Round  I 

As  identified  in  the  Consequence  of  Failure  Playbook  (a  rule  set  developed  by  the  Air 
Force  CE  Career  Field  to  correct  individual  MDI  values),  there  are  “problematic  [categorization 
code]  CATCODEs  whose  MDIs  do  not  accurately  reflect  the  mission  criticality  of  individual 
facilities”  (AFCEC,  2014).  Because  of  this,  the  first  round  focused  on  identifying  the  current 
system’s  issues  and  possible  inputs  into  creating  a  better  MDI.  One  important  step  in  developing 
the  Round  I  questions  was  conducting  a  pilot  study.  Skulmoski  et  al.  (2007)  suggests,  “A  pilot 
study  is  sometimes  conducted  with  the  goals  of  testing  and  adjusting  the  Delphi  questionnaire  to 
improve  comprehension,  and  to  work  out  any  procedural  problems.”  In  accordance  with  this 
literature’s  guidance,  graduate  students  from  the  Air  Force  Institute  of  Technology  studying 
engineering  asset  management  validated  each  question’s  intent  and  clarity.  A  pilot  study  was 
conducted  for  this  round  and  subsequent  rounds  of  the  Delphi  study. 

The  results  from  the  first  round  produced  some  expected  results  but  also  identified 
unexpected  new  themes.  The  panel  agreed  that  the  current  MDI  method  provided  repeatable, 
easy-to-use  values  from  a  common  frame  of  reference  and  that  it  assists  leaders  in  prioritization. 
The  inputs  that  should  be  used  are  identified  in  Table  2. 


Table  2.  Inputs  to  Consider  When  Developing  the  MDI  Value 


Criticality 

Air  Force  Mission 

Local  Mission 

Consequence  of 
Failure 

Mission  Degradation 

Fiscally 

Life,  Health,  and  Safety  Issues 

Redundancy 

Air  Force  Mission 

Local  Mission 

Replaceability 

Asset  Specific 
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One  panel  member  pointed  out  that  Air  Force  mission  criticality  should  be  measured  by 
using  the  Air  Force  core  functions.  The  core  functions  of  the  Air  Force  are  a  set  of  activities  that 
represent  the  full  range  of  the  service’s  capabilities.  These  functions  are  intended  to  be  available 
for  military  (as  well  as  non-military)  operations  around  the  world.  A  list  of  the  core  functions,  as 
well  as  the  Major  Command  (MAJCOM)  responsible  for  providing  them,  is  in  Appendix  F. 

The  panel  identified  a  number  of  issues  associated  with  MDI  as  well.  One  of  the  major 
flaws  was  the  inability  to  properly  identify  non-flightline  mission  assets  and  the  facilities  used  to 
support  them.  Some  panel  members  believed  that  the  inconsistency  came  from  the  usage  of 
CATCODEs  as  the  main  determinant.  The  lack  of  individuality  that  accompanied  the 
CATCODE  method  led  the  panel  to  suggest  that  local  mission  impact  was  not  accounted  for  in 
the  process.  One  panel  member  said,  “Relying  on  the  CATCODE  system  infers  that  it  is  good 
data. .  .1  don’t  believe  that  is  an  accurate  assumption.”  Another  issue  presented  was  that  the 
current  system  does  not  allow  an  adjustment  if  there  are  redundant  assets.  To  be  specific,  all 
assets  receive  the  same  MDI  value  whether  they  are  the  primary  or  secondary  in  support  of  the 
mission.  The  panel  posed  the  question,  “What  level  of  the  mission  is  it  supporting?”  Looking 
from  the  Air  Force  level,  losing  power  to  one  airstrip  may  not  be  an  issue;  but  at  the  local 
mission  level,  it  would  be  important  to  get  the  situation  remedied.  This  point  of  view  also 
revealed  that  only  the  local-level  experts  know  the  second  and  third-order  effects  that  occur 
when  a  specific  asset  is  no  longer  usable. 

The  questionnaire  closed  by  attempting  to  elicit  alternative  methods  for  the  MDI.  Panel 
members  were  asked  in  an  open  response  question:  “Is  there  another  metric  that  should  be  used 
rather  than  the  MDI?”  The  responses  ranged  from  the  “Chief  of  Staff  of  the  Air  Force  priority 
list”  to  the  “Facility  Utilization  Board”  which  is  a  meeting  held  at  each  base  to  detennine  local 
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requirements.  All  of  the  issues  the  panel  addressed  helped  in  creating  the  second  round  of 
questions. 

Delphi  Round  II 

Round  II  was  designed  to  further  clarify  the  themes  from  the  first  round,  in  addition  to 
detennining  if  the  individual  issues  that  were  identified  are  prevalent  throughout  the  enterprise. 
The  majority  of  the  Delphi  team  agreed  that  both  the  local  and  Air  Force  mission  impact  should 
be  taken  into  consideration  when  producing  the  MDI  metric.  All  members  agreed  the 
Consequence  of  Failure  (CoF),  applied  in  prioritization  decisions,  can  be  measured  using  MDI. 
The  majority  of  the  panel  agreed  that  the  MDI  value  should  be  asset-specific.  The  members  in 
favor  of  having  asset-specific  MDI  values  believe  that  it  would  assist  leaders  in  prioritizing 
funding  and  resource  requirements.  The  others  argued  that  collecting  all  of  the  data  required  to 
make  an  asset-specific  MDI  value  would  be  time  and  cost  prohibitive. 

Another  issue  addressed  in  this  round  was  MDI  inflation.  One  panel  member  noted  that 
it  would  be  hard  to  prevent  overall  MDI  inflation  and,  therefore,  hard  to  ensure  portfolio  wide 
consistency.  MDI  inflation  is  currently  taking  place  because  of  the  ability  to  increase  an 
individual  asset’s  MDI  value  without  the  requirement  to  decrease  another  asset’s  MDI  value.  If 
this  process  continues  without  interference,  the  entire  portfolio’s  effective  range  will  reduce, 
along  with  the  decision-making  power  delivered  by  the  MDI.  When  reducing  inflation,  the  panel 
member  was  concerned  that  reducing  inflation  is  counter  to  the  effort  placed  on  receiving 
funding  to  support  assets.  This  paradox  would  encourage  the  local-level  decision-makers  to  not 
reduce  the  MDI  values  of  their  assets.  If  all  other  installations  are  reducing  values  and  one 
installation  does  not,  the  installation’s  local  assets  will  rank  higher  from  a  portfolio-wide 
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perspective  and,  as  a  result,  receive  more  funding.  To  ensure  portfolio- wide  consistency,  a  rule 
set  would  need  to  be  published  to  encourage  all  installations  to  reduce  their  installation’s 
inflation  of  the  MDI. 

The  next  group  of  questions  focused  on  how  to  account  for  redundant  assets.  One  panel 
member  suggested  that  redundancy  should  be  measured  using  money.  Redundancy  defined  as  a 
function  of  money  supports  the  argument:  if  an  asset  costs  $5,  then  have  an  inventory  of 
replacements  and  do  not  place  any  value  on  redundancy.  In  contrast,  if  the  asset  is  a  one-of-a- 
kind  item,  it  might  be  cost  prohibitive  to  build  the  asset  again.  As  a  result  of  being  one-of-a- 
kind,  the  asset  would  receive  a  higher  weight  in  the  redundancy  measure.  Another  member 
suggested  redundancy  should  be  viewed  as  a  function  of  mission  capacity.  If  multiple  assets 
exist  to  ensure  the  installation  can  provide  the  mission  required  by  the  commander  that  must  be 
identified.  A  panel  member  explained  this  by  giving  the  example  of  “two  parallel  runways 
w/similar  characteristics,  the  loss  of  one  [runway]  does  not  drive  mission  failure...  unless  they 
can’t  generate  enough  sorties  w/one  runway.”  Another  panel  member  made  the  suggestion  to 
add  a  coefficient  to  assets  within  the  same  CATCODE.  The  coefficient  would  allow  the 
primary,  secondary,  or  tertiary  asset  MDI  values  to  be  adjusted  reflect  the  amount  of  redundant 
assets  at  the  specified  level.  The  final  suggestion  from  the  panel  was  that  redundancy  should  be 
determined  at  the  local  and  headquarters  levels. 

The  theme  of  replaceability  was  addressed  in  this  round.  The  team  identified  that 
replaceability  focused  on  resources  available.  Money,  labor,  and  time  had  large  roles  in 
detennining  how  replaceable  an  asset  is.  These  three  resources  are  fluid  and  hard  to  identify 
accurately  over  long  periods  in  order  to  use  them  in  an  MDI  value.  The  impact  on  the  different 
mission  set  was  again  recognized  as  an  input  to  replaceability. 
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In  an  effort  to  address  the  disconnect  between  the  local  and  headquarters  level 
viewpoints,  the  panel  members  were  asked  if  the  MAJCOM  priority  adequately  accounted  for 
the  local  mission  in  the  consequence  of  failure  (CoF)  metric.  The  majority  of  the  panel  agreed 
the  local  mission  was  not  accounted  for  by  the  MAJCOM  and  that  the  weightings  should  adjust 
over  time.  A  portion  of  the  group  thought  that  only  the  corrected  MDI  value  should  be  used  as 
the  CoF  metric.  The  other  portion  of  the  panel  suggested  that  the  weightings  should  be  revisited 
on  a  recurring  basis  to  ensure  they  reflect  the  overall  goals  at  the  Air  Force  and  local  levels.  The 
panel  was  asked  if  the  current  process,  which  allows  an  individual  asset’s  MDI  value  to  be 
adjusted  to  a  higher  value,  leads  to  the  MAJCOM  priority  and  MDI  values  combining.  The 
majority  of  the  panel  agreed  that  these  two  inputs  were  not  mutually  exclusive.  The  panel  agreed 
that  having  an  adjustment  factor  to  vary  a  specific  CATCODE  within  a  core  function  would 
correct  the  MDI  values  of  assets  not  supporting  the  flying  mission  set. 

The  final  issue  focused  on  the  application  of  the  MDI  to  utilities  and  infrastructure  that 
affect  numerous  assets.  The  panel  suggested  three  courses  of  action  (COA).  In  COA  1,  for  the 
utility  and  infrastructure  system,  assign  an  MDI  equal  to  the  highest  MDI  for  a  facility  connected 
to  it.  COA  2  suggested  using  a  weighted  sum  or  the  mean  of  all  the  facilities  affected  by  the 
utility  and  infrastructure  system.  A  member  also  addressed  that  when  multiple  facilities  are 
affected,  the  MDI  value  assigned  to  the  project  should  increase  because  of  the  interconnectivity 
of  the  project.  The  final  suggested  COA  was  to  address  situations  on  a  requirement-specific 
basis.  This  COA  would  need  to  have  a  procedure  in  place  to  weight  the  requirements  to  ensure  it 
does  not  get  misused. 

Overall,  round  II  helped  clarify  the  disconnects  discovered  in  the  first  round.  In  the 
development  of  round  II,  the  feedback  from  round  I  implied  there  might  be  a  division  in  the 
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panel  with  one  group  focused  on  local  influence  of  the  MDI  value  and  the  other  group  focused 
on  the  headquarters  influence.  When  analyzing  the  responses  from  round  II,  it  became  clear  that 
there  were  two  groups.  One  group  focused  more  on  the  significance  at  the  local  level  (micro 
viewpoint)  while  the  other  focused  on  the  significance  at  the  headquarters  level  (macro 
viewpoint).  Berman’s  (1978)  findings  suggest  that  the  implementation  of  federal  policy  at  the 
local  level  lends  support  to  the  discovery  of  two  distinct  views:  micro  view  (local  level)  and 
macro  view  (global  level).  Bennan  (1978:32)  believes  this  issue  stems  from,  “micro¬ 
implementation  cannot  be  effective  unless  local  delivery  organizations  undergo  an  adaptive 
process.”  Bennan  suggests  that  without  the  involvement  of  the  local  level  in  the  creation  of  the 
process,  the  local  decision-makers  will  implement  the  policy  to  the  best  of  their  ability. 

However,  this  adaptation  of  the  federal  policy  may  not  accomplish  the  original  intention  of  the 
policy.  The  disconnect  created  when  policy  is  created  at  the  strategic  level  and  applied  at  the 
micro  level  is  difficult  to  remedy  without  compromise.  The  last  round  attempts  to  guide  the 
panel  to  a  consensus  on  the  inputs  that  should  be  addressed  in  the  creation  of  an  asset’s  MDI. 

Delphi  Round  III 

Round  III  began  with  a  question  to  apply  weights  to  all  of  the  values  identified  in  the 
earlier  rounds.  Table  3  shows  the  responses  for  the  weights  from  the  five  Delphi  members  who 
responded  in  Round  III.  Table  3  reveals  several  interesting  findings.  First,  these  results  clearly 
show  the  micro  and  macro  level  focus  groups.  Members  A,  C,  and  D  weighted  the  local  level 
higher  than  the  Air  Force  level,  while  member  E  weighted  the  Air  Force  level  higher.  Panel 
member  B  gave  equal  weighting  to  both  the  micro  and  macro  level.  Second,  although  cost  was 
identified  throughout  the  first  two  rounds,  no  member  placed  any  weight  in  that  category.  This 
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indicates  that  cost  does  not  represent  value  to  the  panel  for  MDI.  Although  this  result  was  not 
initially  expected,  Selart  and  Johansen  (2011)  point  out  that  during  the  decision-making  process, 
solutions  created  by  an  AFT  group  are  often  focused  on  cost  and  solutions  created  by  the  VFT 
group  focus  more  on  long-term  goals.  Third,  redundancy  received  a  weight  from  all  panel 
members-even  the  member  biased  towards  the  micro  view;  this  reinforced  the  importance  of  it 
being  a  valuable  input  to  the  MDI.  Finally,  only  two  members  weighted  “time  to  replace”  and 
“number  quantity  of  missions  impacted.”  With  only  part  of  the  panel  giving  these  categories 
some  consideration,  this  suggests  that  these  values  remain  important,  but  to  a  lesser  degree,  than 
categories  that  received  responses  from  all  panel  members. 


Table  3.  Delphi  Value  Weighting  Response 


Panel  Member 

Values  for  the  MDI  Metric 

A 

B 

c 

D 

E 

Criticality  at  the  Local  level 

30 

40 

50 

90 

25 

Criticality  of  the  Core  Functions 
(Air  Force  level) 

20 

40 

10 

5 

35 

Redundancy 

35 

10 

20 

5 

15 

Time  to  replace 

10 

10 

0 

0 

0 

Cost 

0 

0 

0 

0 

0 

Number  of  Local  Level  Missions 
Impacted 

0 

0 

10 

0 

10 

Number  of  Core  Functions 
Missions  Impacted  (AF  level) 

0 

0 

10 

0 

15 

The  remaining  questions  in  the  round  asked  the  panel  three  things:  what  level  of 
commander  influence  should  be  the  most  important;  should  assets  offered  by  the  local 
community  count  as  redundant;  and  should  the  Air  Force  adapt  the  NAVFAC  methodology.  The 
division  in  the  panel  remained  when  responding  to  whether  a  local  commander’s  influence  or 
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headquarters  commander’s  influence  was  more  valuable.  With  regards  to  redundancy  found  in 
the  local  community,  one  panel  member  brought  up  that  quality  of  life  assets  that  are  available 
off-base  (e.g.,  library)  should  receive  a  reduction  in  their  MDI  value  because  the  asset  is 
redundant.  A  portion  of  the  panel  believed  this  could  be  taken  advantage  of  in  order  to  produce  a 
cost  savings.  The  other  members  of  the  panel  believed  that  Air  Force  installations  should  be 
viewed  as  standalone  and  provide  a  complete  set  of  services  to  the  members.  The  Delphi 
concluded  with  a  question  on  whether  the  NAVFAC  model  should  be  adapted,  and  the  panel  was 
spilt  on  the  question.  One  panel  member  noted  that  rather  than  “Air  Force-ize”  our  system,  it 
would  be  easier  to  adopt  an  existing  method.  Another  believes  the  NAVFAC  method  requires 
too  many  resources  in  the  current  fiscal  environment.  Overall,  round  three  helped  validate  the 
inputs  to  be  placed  into  the  metric’s  system  and  affirm  the  two  outlooks  on  MDI  values. 

Summary  of  Delphi  Study 

The  research  focus  was  to  provide  AFCEC  with  insight  on  how  to  produce  a  correct  MDI 
value  while  mitigating  inflation.  The  Delphi  team  helped  provide  input  in  a  non-attribution 
environment  to  improve  the  metric.  Every  round  aimed  to  validate  and  clarify  the  metric  and 
environment  surrounding  its  creation.  One  limitation  of  these  results  was  that  the  Delphi  group 
decreased  in  size  for  each  round.  Additionally,  due  to  the  audience  required  to  complete  the 
questionnaires,  more  time  was  given  in  each  round  to  receive  inputs,  thus  limiting  the  number  of 
rounds  completed.  Finally,  the  Delphi  group’s  opinions  and  suggestions  are  grounded  in  their 
experience  yielding  a  bias  dependent  on  this  experience.  The  results  should  still  be  used  to 
improve  the  current  conditions  despite  the  accompanying  limitations.  The  following  section 
addresses  how  the  data  collected  through  the  Delphi  were  applied  to  create  a  VFT  model. 
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Value  Focused  Thinking  Hierarchy  Creation 

The  Delphi  study  provided  insight  into  step  one,  problem  identification,  of  the  VFT 
methodology.  This  section  addresses  step  two  through  five  of  perfonning  a  VFT  analysis: 
create  value  hierarchy,  develop  evaluation  measures,  create  value  functions,  and  weight  the  value 
hierarchy  (Shovaik,  2001).  Although  there  are  ten  steps  to  the  VFT  methodology,  only  the  first 
five  are  addressed  because  the  final  five  steps  address  the  implementation  of  the  model.  Due  to 
the  large  number  of  alternatives  the  VFT  model  would  need  to  be  applied  to  (over  130,000),  the 
final  five  steps  were  beyond  the  scope  of  this  research. 

When  creating  a  value  hierarchy,  the  recurring  themes  identified  by  the  Delphi  were 
used.  Specifically,  the  impact  categories  and  performance  measures  identified  in  the  first  two 
rounds  were  used  to  produce  weights  in  round  III.  Three  out  of  four  of  the  impact  categories 
identified  were  ‘ilities.’  de  Week  et  al.  (2012:1)  defined  ‘ilities’  as,  “properties  of  engineering 
systems  that  often  manifest  and  determine  value. . .  rather  than  being  primary  functional 
requirements,  these  properties  concern  wider  system  impacts.”  Revisiting  the  Federal  Real 
Property  Council’s  definition  (2011)  of  MDI,  there  is  no  surprise  that  when  inputs  were  defined  a 
majority  of  them  were  ‘ilities.’  The  first  two  rounds  created  a  list  of  inputs  that  consisted  of 
interruptability,  redundancy,  replaceability,  and  the  number  of  missions  affected.  These  inputs 
were  assigned  as  impact  categories  for  the  hierarchy. 

When  defining  interruptability,  the  same  definition  as  the  NAVFAC  model  was  used, 
“how  long  could  the  ‘functions’  supported  by  your  facility  (functional  element)  be  stopped 
without  adverse  impact  to  the  mission?”  (Antelman,  Dempsey,  &  Brodt,  2008:142).  This 
definition  matched  the  themes  addressed  by  the  Delphi  panel.  The  Delphi  panel  suggested  this 
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‘iltiy’  must  be  addressed  by  the  both  the  local  leadership  and  the  Core  Function  Lead  Integrators 
(CFLI)  to  best  represent  its  impact. 

The  next  impact  category  identified  was  redundancy.  The  Delphi  panel  defined 
redundancy  as,  the  availability  of  other  assets  to  fulfill  the  same  mission.  This  measure  can  be 
applicable  to  secondary  runways  or  any  other  assets  with  backups.  The  panel  suggested  that  this 
be  examined  at  both  the  micro  (local)  level  and  macro  (core  function)  level  to  truly  capture  the 
correct  replaceability.  Half  of  the  panel  agreed  that  redundancy  of  the  assets  should  be  taken 
into  account  at  the  micro  (local)  level,  while  the  other  half  of  the  panel  believed  that  redundancy 
should  be  applied  at  the  macro  (core  function)  level.  In  particular,  the  quality  of  life  facilities 
were  brought  up  when  looking  at  the  micro  viewpoint  as  an  opportunity  to  help  alleviate  some  of 
the  fiscal  stress  associated  with  the  operations  and  maintenance  of  these  facilities. 

The  third  impact  category  identified  was  replaceability.  The  Delphi  panel  suggested  that 
replaceability  be  measured  as  the  time  required  to  replace  that  asset  or  provide  a  work-around. 
This  is  important  because  if  the  asset  can  be  repaired  in  a  matter  of  hours,  there  is  not  as  big  of 
an  impact  to  mission-capable  level  as  when  an  asset  may  take  months  to  repair. 

The  final  impact  category  identified  was  the  number  of  missions  affected  when  an  asset 
fails.  This  impact  category  addresses  how  interconnected  an  asset  is  to  various  mission  sets. 

The  panel  felt  that  the  number  of  missions  at  the  local  installation,  as  well  as  the  core  functions, 
must  be  addressed  to  characterize  the  asset’s  interconnectivity  at  the  micro  and  macro  levels. 

The  impact  categories  and  performance  measures  were  developed  in  tandem.  The  impact 
categories  are  shown  in  the  first  level  of  the  hierarchy  in  Figure  16.  The  second  level  of  the 
hierarchy  represents  the  perfonnance  measures  identified  by  the  Delphi  study  in  round  III.  The 
perfonnance  measures  were  defined  following  the  impact  categories  to  ensure  all  of  the  impact 
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categories  were  measurable  after  the  panel  agreed  on  them.  Figure  16  represents  the  end  of  steps 
two  and  three  of  the  VFT  method  used  to  create  the  value  hierarchy  and  develop  performance 
measures. 
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Figure  16.  VFT  MDI  Hierarchy 


After  the  performance  measures  were  developed,  the  fourth  step  of  creating  value 
functions  took  place.  Value  functions  “combine  the  multiple  evaluation  measures  into  a  single 
measure  of  the  overall  value  of  each  evaluation  alternative”  (Kirkwood,  1997:53).  By  creating 
value  functions,  the  effect  of  how  much  each  performance  measure  contributes  to  the  overall 
scoring  can  be  determined.  Each  value  function  was  developed  through  feedback  from  the 
Delphi  panel  and  adaptation  from  the  literature  review. 

For  the  performance  measures  in  the  interruptability  category,  the  NAVFAC  scale  was 
used  to  demonstrate  how  long  the  mission  set  could  not  be  impacted  with  the  asset’s  failure. 
This  is  a  categorical  scale  with  four  categories  (Antelman,  Dempsey,  &  Brodt,  2008: 142).  The 
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same  scale  was  used  for  both  the  local  and  CFLI  level  decisions  with  the  values  of  each  category 
as  shown  in  Figure  17.  The  values  and  definition  for  each  level  of  the  scale  are: 


•  Immediate  (1.0)  (Any  interruption  will  immediately  impact  mission  readiness) 

•  Brief  (0.75)  (minutes  or  hours  not  to  exceed  24  hours) 

•  Short  (0.5)  (Days  not  to  exceed  7  days) 

•  Prolonged  (0.25)  (more  than  a  week) 


Figure  17.  Value  Function  for  Interruptability 


Replaceability  only  contained  one  performance  measure.  The  measure  that  was 
developed  was  the  time  required.  In  an  effort  to  ease  the  use  of  the  model  for  the  Air  Force  CE 
career  field,  the  same  time  scale  as  presented  in  the  interruptability  category  was  used.  However, 
it  must  be  noted  that  replaceability  is  measuring  the  time  it  would  take  to  bring  that  asset  back  up 
to  its  capability.  In  addition,  the  points  associated  with  the  scale  are  reversed.  The  reversal 
happens  because  if  the  asset  can  immediately  be  replaced,  it  is  not  as  imperative  as  an  asset  that 
will  take  more  than  a  week  to  be  replaced.  Figure  18  shows  the  values  used,  and  the  definitions 
and  points  assigned  are: 
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•  Immediate  (0.25)  (Any  interruption  will  immediately  impact  mission  readiness) 

•  Brief  (0.5)  (minutes  or  hours  not  to  exceed  24  hours) 

•  Short  (0.75)  (Days  not  to  exceed  7  days) 

•  Prolonged  (1.0)  (more  than  a  week) 
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Figure  18.  Value  Function  of  Replaceability 


The  redundancy  perfonnance  measures  are  addressed  at  both  the  local  and  core  function 
level.  The  evaluation  measure  used  for  these  performance  measures  was  selected  as  a  decreasing 
continuous  function.  Kirkwood  (1997)  recommends  that  if  preferences  decrease  over  the  x- 
direction,  then  the  appropriate  value  function  is: 


V(x)  = 


1  -  exp  [-  (High  -  x)  /  p] 

1  -  exp  y-^High  -  Low )  /  p] 


(4) 


In  this  equation,  p  represents  the  exponential  constant  to  shape  the  disutility  curve.  High  and  low 
are  the  minimum  and  maximum  values  of  the  range.  The  p  value  selected  for  this  application 
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was  -5  to  show  the  diminishing  utility  of  having  more  redundant  assets.  This  was  modified  from 
the  value  function  titled  “complexity  of  contingencies”  by  Karydas  and  Gifun  (2006).  The  high 
and  low  values  selected  for  were  5  and  0,  respectively.  The  range  was  small  because  any  asset 
that  has  more  than  5  redundant  partners  is  not  as  critical  as  a  one-of-a-kind  asset;  therefore,  the 
value  in  this  performance  measure  is  zero.  At  the  CFLI  level,  this  may  result  in  many  assets 
receiving  0  for  the  performance  measure,  but  this  helps  identify  one-of-a-kind  assets  in  the  Air 
Force.  The  weights  associated  with  each  integer  in  the  value  function  were  matched  as  closely 
as  possible  to  the  values  identified  by  the  Delphi  panel.  Figure  19  shows  the  value  function  and 
Table  4  shows  the  values  resulting  from  the  equation  with  the  given  assumptions. 


Redundant  Assets 

Figure  19.  Value  Function  of  Redundancy 


Table  4.  Redundant  Assets  Weights 


Redundant 

Assets 

Weight 

0 

1.00 

1 

0.71 

2 

0.48 

3 

0.29 

4 

0.13 

5 

0.00 
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The  “number  of  missions  affected”  category  also  has  two  performance  measures.  Both  the  local 
and  Air  Force  level  mission  sets  were  taken  into  account  in  this  category.  Adopting  the  law  of 
diminishing  returns  applied  by  the  NAVFAC  model,  these  measures  used  an  increasing 
preference  evaluation  measure  with  a  p  =  5  to  show  the  overall  impact.  The  minimum  and 
maximum  values  were  0  and  10,  respectively.  The  equation  for  increasing  preferences  is  shown 
in  equation  5  (Kirkwood,  1997).  The  resulting  weights  are  shown  in  Figure  20  and  Table  5. 


V(x)  = 


1-exp  [-(x  -  Low)  !  p] 

1  -  exp  [-(High  -  Low)  /  p] 


(5) 


Number  of  Assets  Affected 


Figure  20.  Value  Function  of  Missions  Affected 
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Table  5.  Number  of  Mission  Sets  Affected  Weights 


#  of  Mission 
Sets  Affected 

Weight 

0 

0.00 

1 

0.21 

2 

0.38 

3 

0.52 

4 

0.64 

5 

0.73 

6 

0.81 

7 

0.87 

8 

0.92 

9 

0.97 

10 

1.00 

With  all  of  the  perfonnance  measures  defined  and  value  scales  developed,  the  fifth  step 
of  the  VFT  process  is  to  give  weights  to  the  value  hierarchy.  During  the  final  round  of  the 
Delphi,  each  panel  member  was  asked  to  give  weights  to  these  categories.  The  panel  members 
remained  divided  in  the  mindsets  of  micro  (local)  and  macro  (headquarters)  viewpoints.  Once 
the  panel  members  were  split  into  their  respective  group  based  off  their  open  responses  and 
weighting,  each  category  (micro  and  macro)  was  averaged.  One  of  the  panel  members  only  used 
95  out  of  the  possible  100  points  for  weights,  so  the  response  was  standardized  to  a  scale  equal  to 
the  remainder  of  the  responses.  The  average  weights  for  each  group  are  shown  in  Figure  21  and 
Figure  22. 
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Figure  2 1 .  Micro  Viewpoint  VFT  Hierarchy  Weights 
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Figure  22.  Macro  Viewpoint  VFT  Hierarchy  Weights 
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The  resulting  weights  have  the  replaceability  weight  in  common  and  the  numbers  of 
missions  affected  were  very  close.  The  installation-level  point  of  view  (micro  view)  yielded  a 
model  in  which  60%  of  the  MDI  value  comes  from  local  decision-makers  and  35%  are  managed 
at  the  CFLI  level.  The  HAF  point  of  view  (macro  view)  resulted  in  a  more  balanced  model,  with 
the  base  accounting  for  44%  of  the  MDI  value  and  HAF  accounting  for  51%  of  the  model. 

These  weights  are  shown  in  Figure  23  and  Figure  24. 
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Figure  23.  Micro  Viewpoint  Leadership  Influence 
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Figure  24.  Macro  Viewpoint  Leadership  Influence 

In  an  effort  to  reduce  bias  toward  the  micro  (local)  and  macro  (headquarters)  viewpoints, 
one  additional  model  was  created.  This  model  used  the  feedback  from  all  panel  members  with 
the  exception  of  the  one  who  was  extremely  biased  toward  the  local  level.  The  weights  are  the 
mean  value  assigned  to  each  category  by  the  panel.  Because  the  micro  and  macro  groups  were 
equal,  the  final  weights  are  averages  and  represent  an  attempt  to  reduce  the  groups’  biases.  The 
new  hierarchy  and  weights  are  shown  in  Figure  25,  and  the  combined  leadership  bias  is  shown  in 
Figure  26. 
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Figure  25.  Combined  Views  Hierarchy  Weights 
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Figure  26.  Combined  Views  Leadership  Influence 
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One  of  the  main  benefits  of  the  VFT  methodology  is  that  once  the  categories  and 
measurements  are  defined,  the  weighting  of  the  hierarchy  and  the  perfonnance  measure  scales 
can  be  adapted  as  leadership  detennines.  The  models  developed  in  this  chapter  are  meant  to  be 
applied  at  the  individual  asset  level;  however,  the  models  could  also  be  applied  to  the 
CATCODE  level  if  the  generalizations  of  assets  that  occur  when  using  CATCODEs  are  accepted 
by  the  decision-makers.  The  following  chapter  will  address  the  conclusions  and 
recommendations  of  the  research. 
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V.  Conclusions  and  Recommendations 


Throughout  the  Delphi  study,  the  panel  reaffirmed  the  importance  of  having  a  method  to 
measure  the  consequence  of  failure  of  an  asset.  The  questionnaire  confirmed  that  currently  the 
Air  Force  Civil  Engineer  (CE)  community  struggles  with  how  to  assign  the  Mission  Dependency 
Index  (MDI)  values  to  assets.  Despite  this  struggle,  the  data  collected  from  the  Delphi  study  and 
applied  to  the  Value  Focused  Thinking  (VFT)  methodology  produced  three  hierarchies  to  assist 
in  assigning  asset-specific  MDI  values.  This  section  answers  the  investigative  questions, 
addresses  the  study’s  conclusions,  discusses  the  study’s  significance,  and  offers  a  few 
recommendations  for  future  research. 

Investigative  Questions  Answered 

This  section  revisits  the  investigative  questions  and  applies  what  was  learned  through  the 
Delphi  process.  The  first  investigative  question  was,  what  are  the  inputs  needed  to  define  the 
MDI  value  of  a  building?  The  Delphi  team  identified  the  values  associated  with  MDI  to  be 
interruptability,  redundancy,  replaceability,  and  the  number  of  missions  affected  when  a  specific 
asset  fails.  These  inputs  should  all  be  addressed  when  creating  the  MDI  metric  for  an  asset. 

The  second  question  was,  what  weight  should  be  applied  to  each  of  the  inputs  when 
assigning  an  MDI  value  to  a  facility  or  asset?  The  study  revealed  a  split  in  perspectives  among 
the  leaders  interviewed.  As  a  result  of  the  divide,  three  models  were  produced:  a  micro  (local) 
viewpoint,  a  macro  (headquarters)  viewpoint,  and  a  model  that  combined  the  viewpoints.  The 
weights  were  not  agreed  upon  through  the  Delphi  process  due  to  time  restrictions  of  the  study. 
Resolving  this  disagreement  is  important  to  create  a  model  that  the  entire  CE  career  field  can 
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support  and  employ.  Detennining  one  set  of  weights  for  each  input  category  and  the 
perfonnance  measures  is  an  area  for  possible  future  research.  In  addition  to  creating  an  agreed 
upon  set  of  weights,  the  value  functions  that  were  adapted  from  the  literature  could  be  validated 
through  leadership  or  by  performing  another  questionnaire.  Each  value  function  can  be  adapted 
and  changed  to  represent  the  decision-maker’s  desires. 

The  third  question  asked,  what  is  the  desired  distribution  of  the  MDI  values?  A  portion 
of  this  question  was  answered  in  round  I  of  the  Delphi.  The  majority  of  the  Delphi  panel  agreed 
that  the  range  should  be  0-100.  The  panel  recognized  that  identifying  the  distribution  of  values 
is  counter  to  the  process,  and  the  distribution  should  be  validated  after  all  alternatives  have  been 
applied  to  the  model. 

The  final  question  investigated  was,  according  to  CE  subject  matter  experts  (SME),  how 
should  the  MDI  be  applied  to  the  prioritization  of  infrastructure  funding  decisions?  All  panel 
members  agreed  MDI  values  need  to  be  included  in  the  prioritization  of  funding  decisions. 

When  the  metric  is  applied  to  prioritization,  each  situation  needs  to  be  focused  on  the 
requirement  of  the  asset.  The  panel,  regardless  of  their  bias,  identified  that  the  importance  to 
both  the  micro  and  macro  levels  needs  to  be  taken  into  consideration.  The  bias  of  the  members 
led  to  different  weight  being  assigned  to  capture  the  influence  of  the  two  levels.  In  brief,  each 
investigative  question  was  addressed  in  the  Delphi  rounds  in  an  effort  to  answer  the  research 
questions. 

Conclusions  and  Significance  of  Research 

As  the  Delphi  study  addressed  each  investigative  question,  the  primary  research  question 
can  be  addressed:  how  can  the  Air  Force  Civil  Engineer  Center  (AFCEC)  create  a  process  to 
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prevent  MDI  value  inflation  while  maintaining  a  usable  metric  for  funding  decisions.  A  review 
of  the  history  of  the  current  MDI  definition  created  in  2008  revealed  that  it  was  meant  as  a 
stopgap  measure  until  a  more  reasoned  method  was  produced.  Since  2008,  efforts  have  been 
made  by  the  CE  career  field  to  create  internal  processes  to  correct  individual  errors,  but  the 
underlying  system  issues  remain.  The  biggest  issue  is  that  the  current  system  has  no  way  of 
calculating  an  MDI  value  of  a  building  without  comparing  it  to  alternatives.  This  issue  can  be 
minimized  if  the  impact  categories  and  performance  measures  created  by  the  CE  leadership  and 
SMEs  in  the  Delphi  process  are  taken  into  account.  This  research  provides  AFCEC  with  three 
models  it  can  choose  to  implement,  the  micro  perspective,  the  macro  perspective,  or  the 
combined  perspectives,  to  create  asset-specific  MDI  values  that  do  not  require  comparing  it  to 
alternatives  throughout  the  Air  Force. 

The  resulting  values  identified  by  the  Delphi  panel  were  similar  to  the  Naval  Facilities 
Engineering  Command  (NAVFAC)  model.  This  is  interesting  because  the  NAVFAC  model  was 
not  introduced  to  the  panel  until  the  last  question  of  the  final  round.  The  values  identified  in  the 
NAVFAC  model  were  interruptability,  relocateability,  replaceability,  and  number  of  missions 
affected.  The  model  produced  in  this  research  addresses  all  of  those  values  except 
relocateability,  as  well  as  suggesting  that  a  new  value  of  redundancy  should  be  incorporated.  In 
the  combined  viewpoint  model,  redundancy  accounted  for  20  percent  of  the  overall  input  when 
creating  the  MDI  value.  The  NAVFAC  model  focused  on  the  micro  (local)  level  for  84%  of  the 
value  and  the  micro  viewpoint  group  in  the  panel  suggested  that  local  leaders  should  have  60% 
of  the  input  into  creating  the  MDI.  These  similarities  of  methodologies  demonstrate  that  two 
different  efforts  created  common  values  when  creating  a  metric  for  the  consequence  of  failure  of 
infrastructure. 
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During  this  exploratory  study,  another  effort  was  made  by  the  CE  career  field  to  solidify 
the  MDI  metric.  This  demonstrates  the  recognition  that  the  current  system  used  to  create  MDI 
values  needs  to  be  more  flexible  for  varying  mission  sets  in  addition  to  being  more  reproducible 
given  the  same  constraints  in  two  situations.  This  additional  effort,  accomplished  by  contract, 
involved  representatives  from  every  major  command  (MAJCOM),  some  base  level  experts,  and 
individuals  from  AFCEC  and  headquarters  Air  Force.  The  effort  identified  the  same  issues  as 
the  Delphi  team.  The  contract  effort  created  another  method  to  adjust  individual  category  codes 
(CATCODE)  MDI  values;  however,  it  only  fueled  the  inflation  issue  since  it  did  not  include  the 
redistribution  of  values  when  an  individual  is  granted  a  higher  MDI  value. 

Recommendations  for  Action  and  Future  Research 

Using  the  data  and  analysis  gathered  from  this  research,  additional  research  could  be 
pursued  to  further  explore  the  issues  for  which  the  Delphi  team  could  not  reach  a  consensus  over. 
The  first  area  that  could  be  examined  is  finding  a  balance  between  the  micro  (local)  level 
perspective  and  macro  (higher  headquarters)  level  perspective.  The  second  area  could  use  the 
existing  values  and  evaluate  the  system  using  the  analytical  network  processing  (ANP)  method 
outlined  by  Cheng  and  Li  (2005).  Cheng  and  Li’s  process  involves  creating  a  network  model 
that  uses  qualitative  and  quantitative  data  to  create  decision  possibilities  based  on  user-defined 
objective  functions.  Leveraging  qualitative  data  collected  in  2008  using  the  NAVFAC 
methodology  at  two  bases,  the  ANP  method  could  be  applied  to  see  if  the  models  recommended 
in  this  research  reflect  leadership’s  desires.  Another  recommended  area  for  additional  research 
would  be  to  examine  how  the  MDI  could  be  applied  in  an  expeditionary  setting  that  becomes 
more  pennanent.  Recently,  the  presence  in  the  Middle  East  has  reduced,  but  the  Air  Force 
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desires  to  maintain  a  number  of  bases  in  a  more  permanent  posture.  As  the  infrastructure 
changes  from  expeditionary  to  pennanent,  the  funding  methods  may  change  as  well.  The  final 
recommendation  is  to  acquire  the  required  data  from  an  existing  installation  and  apply  the  model 
presented  with  the  two  varying  weights  to  see  how  the  model  perfonns  and  how  it  can  be 
adapted  to  accurately  capture  real-world  conditions. 

Summary 

This  research  began  as  an  effort  to  correct  the  MDI  system  the  Air  Force  employs  and 
identify  the  inputs  that  should  be  taken  into  consideration.  During  the  process  of  identifying  the 
inputs,  other  areas  for  improvement  outside  the  scope  of  this  study  were  revealed.  Despite  the 
inability  to  converge  on  what  level  the  MDI  represents,  micro  or  macro,  two  models  were 
created  in  an  effort  to  show  each  perspective.  Overall,  using  this  research,  the  CE  career  field 
can  improve  the  current  methodology  and,  more  importantly,  continue  to  improve  the  MDI 
effectiveness  in  the  prioritization  of  projects  for  funding. 
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Appendix  A.  NAVFAC  Survey  Questions 

(Antelman,  Dempsey,  &  Brodt,  2008) 

Question  #1:  How  long  could  the  "functions"  supported  by  your  facility  (functional  element)  be 
stopped  without  adverse  impact  to  the  mission? 

•  Immediate  (any  interruption  will  immediately  impact  mission  readiness), 

•  Brief  (minutes  or  hours  not  to  exceed  24  hours), 

•  Short  (days  not  to  exceed  7  days),  or 

•  Prolonged  (more  than  a  week). 

Question  #2:  If  your  facility  was  no  longer  functional,  could  you  continue  perfonning  your 
mission  by  using  another  facility,  or  by  setting  up  temporary  facilities?  (Are  there  workarounds?) 

•  Impossible  (an  alternate  location  is  not  available), 

•  Extremely  Difficult  (an  alternate  location  exists  with  minimally  acceptable  capabilities, 
but  would  require  either  a  significant  effort  (money/man-hours),  dislocation  of  another 
major  occupant,  or  contracting  for  additional  services  and/or  facilities  to  complete), 

•  Difficult  (an  alternate  location  exists  with  acceptable  capabilities  and  capacity  but 
relocation  would  require  a  measurable  level  of  effort  (money /man-hours),  but  mission 
readiness  capabilities  would  not  be  compromised  in  the  process), 

•  Possible  (an  alternate  location  is  readily  available  with  sufficient  capabilities  and 
capacity,  in  addition  the  level  of  effort  has  been  budgeted  for  or  can  be  easily  absorbed). 

Question  #3:  How  long  could  the  services  provided  by  (named  organizational  subcomponent) 
be  interrupted  before  impacting  your  mission  readiness? 

•  Immediate  (any  interruption  will  immediately  impact  mission  readiness), 
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Brief  (minutes  or  hours  not  to  exceed  24), 


•  Short  (days  not  to  exceed  7  days),  or 

•  Prolonged  (more  than  a  week  or  there  are  more  than  sufficient  redundancies  or  there  is 
a  known  quantity  of  excess  capacity  available  in  the  foreseeable  future). 

Question  #4:  How  difficult  would  it  be  to  replace  or  replicate  the  services  provided  by  (named 
organizational  subcomponent)  with  another  provider  from  any  source  before  impacting  the 
command’s  mission  readiness? 

•  Impossible  (there  are  no  known  redundancies  or  excess/surge  capacities  available,  or 
there  are  no  viable  commercial  alternatives, 

•  Extremely  Difficult  (there  are  minimally  acceptable  redundancies  or  excess/surge 
capacities  available,  or  there  are  viable  commercial  alternatives,  but  no  readily 
available  contract  mechanism  in  place  to  replace  the  sendees), 

•  Difficult  ( services  exist  and  are  available,  but  the  form  of  delivery  is  ill  defined  or  will 
require  a  measurable  and  unbudgeted  level  of  effort  to  obtain  (money /man -hours),  but 
mission  readiness  capabilities  would  not  be  compromised  in  the  process), 

•  Possible  ( services  exist,  are  available,  and  are  well  defined). 
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Appendix  B.  MIT  Impact  Categories  and  Performance  Measure  Definitions 

(Karydas  &  Gifun,  2006:89) 


Definitions,  impact  categories  and  performance  measures 


Complexity  of  contingencies 


Coordination  with  policies, 
programs,  and  operations 

Economic  impact  of  the 
project 

External  public  image 


Impact  on  the  environment 


Impact  on  health,  safety,  and 
the  environment 
Impact  on  people 


Impact  on  property,  aca¬ 
demic.  and  institute  oper¬ 
ations 

Impact  on  public  image 
Intellectual  property  damage 


Internal  public  image 


Interruption  of  academic 
activities  and  operations 

Interruption  time 

Loss  of  cost  savings 


Physical  property  damage 


frog  rams  affected  by  the 
project 


Minimize  the  cost  of  contingency  arrangements  necessary  for  the  continuity  of  academic  activities  and  operations  of  the 
institute. 

Performance  measure:  The  monetary  cost  of  contingency  arrangements  necessary  to  ascertain  the  continuity  of  academic 
activities  and  operations  for  die  restoration  period. 

Consider  the  degree  of  association  of  the  proposed  project  with  the  academic  (teaching  and  research)  and  business 
objectives  of  die  Institute  and  minimize  die  impact  that  its  delayed  completion  may  have  in  terms  of  public  image, 
academic  program  budgets  and  number  of  students  invoked  and  employed  in  the  associated  program 
Evaluate  the  proposed  project  considering  die  economic  impact  that  a  delayed  completion  may  have  in  terms  of  the 
physical  damage  of  real  and  intellectual  property,  disruption  of  continuity  of  institute  operations  and  wasted  moneys 
representing  added  costs  of  condition-induced  deterioration  and  lack  of  modernization-induced  efficiencies. 

Minimize  die  impact  of  (a)  the  delayed  completion  of  die  project  and  (b)  events  associated  with  this  delay  on  the  image  of 
die  Institute  held  by  parents  of  prospective  students,  prospective  students,  granting  agencies,  donors,  and  regulatory 
agencies. 

terformance  measure:  degree  of  die  negative  image  held  by  parents  of  prospective  students,  prospective  students, 
granting  agencies,  donors,  and  regulatory  agencies. 

Minimize  die  impact  on  die  environment  from  hazards  associated  widi  deficiencies  that  will  be  corrected  with  the 
proposed  project. 

Performance  measure:  The  severity  of  environmental  damage  caused  by  events  associated  widi  delayed  completion  of 
die  proposed  project.  Impact  on  die  environment,  applies  to  die  environment  outside  of  campus  buildings  and  impacts 
that  could  occur  in  the  utility  systems  beyond  the  projection  of  die  exterior  facade  of  any  buildings. 

Evaluate  proposed  project  considering  risk  reduction  opportunities  introduced  by  the  project's  completion.  Minimize 
risk  to  people  and  the  environment  by  correcting  deficiencies  associated  widi  die  proposed  project. 

Minimize  die  impact  on  students,  faculty  and  die  public  from  perils  associated  with  deficiencies  diat  will  be  corrected 
widi  die  proposed  project. 

Performance  measure:  death,  injury,  and  illness  on  individuals  affected  by  die  delayed  completion  of  the  proposed 
project. 

Minimize  the  impact  on  property  (buildings  and  equipment  and  intellectual  property)  from  damage  associated  widi  the 
delayed  completion  of  the  proposed  project.  Minimize  business  interruption  caused  by  events  associated  with  the  delayed 
completionof  die  proposed  project,  i.e.  ascertain  die  continuity  of  academic  activities  (teaching  and  research)by  making 
appropriate  contingency  arrangements. 

Minimize  die  impact  on  die  positive  image  diat  die  institute  strives  to  maintain  toward  the  community,  parents,  business 
partners,  sponsors,  regulatory  agencies,  and  local  government. 

Minimize  the  impact  on  intellectual  and  intangible  property  from  damage  associated  widi  die  delayed  completion  of  the 
proposed  project. 

Performance  measure:  degree  of  'replaceability*  of  affected  property  associated  with  the  delayed  completion  of  die 
proposed  project. 

Minimize  die  impact  of  (a)  the  delayed  completion  of  die  project  and  (b)  events  associated  with  this  delay  on  die  image  of 
die  Institute  held  by  parents  of  existing  students,  students,  faculty,  staff,  and  other  members  of  the  MIT  community. 
Performance  measure:  degree  of  the  negative  image  heldby  parents  of  existing  students,  students,  faculty,  staff,  and  other 
members  of  the  MIT  community. 

Minimize  the  impact  on  the  continuity  of  academic  activities  (teaching,  research,  and  other  supporting  activities,  such  as 
work  environment  or  living  accommodations)  where  appropriate  contingency  arrangements  are  necessary  for  die  period 
necessary  to  restore  normal  operations. 

Minimize  die  length  of  interruption  time  of  academic  activities  and  other  institute  operations. 

Performance  measure:  the  length  of  time  needed  to  restore  academic  activities  and  operations. 

Minimize  the  loss  of  cost  savings  associated  widi  the  delayed  completion  of  the  project  until  unacceptable  deterioration 
or  damage  occurs  or  excessive  additional  cost  is  involved.  Also,  consider  possible  Inst  cost  savings,  which  otherwise 
might  be  obtained  with  the  introduction  of  new  technologies,  higher  efficiency,  and  innovative  design  associated  widi  die 
proposed  project. 

Performance  measure:  die  amount  of  savings,  as  the  difference  between  the  current  cost  and  the  cost  associated  with  the 
delayed  completion  of  the  proposed  project  (when  irreversible  damage  may  occur).  Also  die  additional  amount  of 
savings  associated  widi  die  implementation  of  new  technologies  or  efficient  design. 

Minimize  impact  on  property  (land,  buildings,  and  equipment )  from  damage  associated  with  the  delayed  completion  of 
die  proposed  project.  Performance  measure:  cost  of  restoration  of  affected  property  associated  with  the  delayed 
completion  of  the  proposed  project. 

Minimize  die  inpact  on  academic  program  budgets,  the  number  of  students  involved  and  employed  in  the  associated 
program  and  the  institute's  business  objectives  associated  widi  die  delayed  completion  of  the  proposed  project. 
Performance  measures:  budget  amount  of  academic  program  or  operation,  die  number  of  affected  students,  or  both. 
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Appendix  C.  VFT  Description 


A  Proposed  Value  Focused  Thinking  Study  for: 

Mission  Dependency  Index  Metric 

Primary  Researcher:  Captain  Matthew  Nichols 
United  States  Air  Force,  Air  Force  Institute  of  Technology 

VALUE  FOCUSED  THINKING  STUDY  GOALS 

The  purpose  of  this  Value  Focused  Thinking  (VFT)  study  is  to  provide  Air  Force  Civil  Engineer 
Center  (AFCEC)  with  a  method  to  create  and  apply  the  Mission  Dependency  Index  (MDI)  metric 
currently  used  in  the  prioritization  of  United  States  Air  Force  Civil  Engineer  (CE)  projects  while 
preventing  inflation  of  the  values. 

This  study  will  address  the  following  questions: 

What  inputs  are  taken  into  consideration  when  detennining  an  MDI  value? 

What  is  the  desired  distribution  of  the  MDI  values  to  ensure  the  appropriate  application  of  the 
metric? 

How  should  the  MDI  be  obtained/assigned  to  a  facility? 

VALUE  FOCUSED  THINKING  BACKGROUND  AND  COMPOSITION 
MDI  Value  Information 

The  MDI  values  currently  assigned  to  buildings  where  assigned  because  of  the  Categorization 
Code  (CATCODE).  NAVFAC  provided  CE  with  the  data  produced  through  their  survey 
methodology  and  the  data  was  adapted  to  the  Air  Force’s  infrastructure.  During  this  adaptation, 
the  conservative  values  for  each  CATCODE  were  taken  to  mitigate  any  risk.  Over  the  life  of  the 
metric,  the  MDI  values  of  specific  CATCODES  have  been  adjusted  by  MAJCOMs  and  AFCEC. 
This  course  of  action  has  lead  to  an  inflation  issue  with  MDI  values. 

Value  Focused  Thinking  Background 

A  VFT  study  is  a  methodology  used  for  gathering  data  about  topics  of  processes  to  generate  a 
model  that  ties  strategic  goals  to  routine  operations.  VFT  pre-establishes  the  different  input 
values  applied  in  organizational  decision-making  instead  of  Alternative  focused  thinking,  the 
process  of  selecting  the  strongest  alternative.  The  study  is  conducted  by  selecting  a  panel  of 
experts  in  a  specific  field  and  engaging  them  in  a  guided,  anonymous  discussion  by  asking  the 
group  specific,  pointed  questions.  Through  multiple  rounds  of  questions,  expert  opinions  are 
gathered  and  discussed  amongst  the  group  in  an  effort  to  garner  consensus  and/or  highlight 
differences  in  opinions  on  the  topic  of  discussion.  Throughout  the  iterative  process,  a  model  is 
created  to  identify  and  maximize  decision  opportunities. 


Panel  Composition 
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The  desired  panel  will  be  composed  of  10  -  20  individuals.  The  target  group  of  members  should 
be  subject  matter  experts  or  senior  leaders  in  the  CE  career  field.  This  combined  group  will 
create  an  overall  panel  capable  of  generating  a  clearly  defined  MDI  metric  for  use  in  the 
prioritization  model. 

Air  Force  Civil  Engineer  Senior  Leader  Expertise 

This  group  of  experts  will  consist  of  5-10  individuals  each  of  whom  should  meet  the  following 
qualifications: 

Minimum  15  year  of  experience  managing  infrastructure  assets 
Working  knowledge  of  the  CE  Prioritization  model 

Air  Force  Civil  Engineer  Subject  Matter  Expert  Expertise 

This  group  of  experts  will  consist  of  5-10  individuals  each  of  whom  should  meet  the  following 
qualifications: 

Minimum  3  years  of  experience  prioritizing  infrastructure  asset  projects 
Working  knowledge  of  the  CE  Prioritization  model 

Specific  knowledge  of  CE  funding  and  prioritization  actions  in  support  of  public  infrastructure 

Participation  Requirements  for  All  Panel  Members 

The  panel  will  be  conducted  electronically  via  email.  This  will  afford  panel  members  flexibility 
and  anonymity  in  participation.  All  panel  participants  will  be  requested  to  participate  in  the 
study  by  providing  the  following  input: 

A  brief  demographic  questionnaire  before  the  study  begins 

Minor  additional  input  as  requested  for  study  approval  by  the  Institution  Review  Board 
2-4  rounds  of  discussion.  Each  round  will  consist  of: 

A  brief  list  of  multiple  choice  questions 

Several  open  ended  questions  and/or  an  open  comments  section 

Results  and  input  from  all  participants  for  the  previous  round  (except  first  round) 

Each  round  is  anticipated  to  require  no  more  than  20  minutes  of  participant  time 
Optional:  review  of  final  round  conclusions 

NOTIONAL  VALUE  FOCUSED  THINKING  SCHEDULE 

This  study  is  expected  to  run  from  1  September  2014  through  31  December  2014.  A  notional 
timeline  for  the  study  is  presented  in  Table  1.  The  bold  items  in  Table  1  indicate  items  which 
will  require  panel  member  participation. 
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Table  1.  Notional  Schedule 


Scheduled  Item 

Date  (2014) 

Installation  Review  Board  (IRB)  approval  process 

1-22  September 

Draft  Value  Focused  Thinking  questions  for 

Round  1 

1-22  September 

Select  panel  members 

25-29  September 

Solicit  panel  member  demographic  information 

25-29  September 

Round  1 

1-8  October 

Compile  Round  1  responses,  draft  Round  2 
questions 

8  October  -  1 5  October 

Round  2 

15  -  29  October 

Compile  Round  2  responses,  draft  Round  3 
questions 

29  October  -  6  November 

Round  3 

6  November  -  20 

November 

Compile  Round  3  responses,  draft  final 
conclusions 

20  -  27  November 

Distribute  final  conclusions  to  participants 

6  December 

CONTACT  INFORMATION 

Should  any  questions  arise  about  the  study  or  its  goals  or  execution,  please  contact  the  primary 
researcher  directly  at  any  of  the  following: 

Matthew  Nichols 

Captain,  United  States  Air  Force 
Student,  Air  Force  Institute  of  Technology 
matthew.nichols@afit.edu  or  nichols.matthew.i@gmail.com 
(303) 895-5580 
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Appendix  D.  Round  I  Delphi  Questions 


Mission  Dependency  Index  Metric: 

A  Value  Focused  Thinking  Study 
Primary  Researcher:  Captain  Matthew  Nichols 
United  States  Air  Force,  Air  Force  Institute  of  Technology 

Questions  for  Round  One  of  the  Value  Focused  Thinking  Study 

1)  What  does  the  Mission  Dependency  Index  (MDI)  Value  represent  in  an  infrastructure 
portfolio? 

Please  provide  a  definition  in  your  own  words. 

2)  What  are  the  strengths  of  using  current  MDI  values? 

3)  What  are  the  weaknesses  of  using  the  current  MDI  values? 

4)  What  inputs  need  to  be  taken  into  consideration  when  determining  the  MDI  value  of  a 
facility?  Please  provide  at  least  three  inputs. 

1. 

2. 

3. 

5)  What  are  the  advantages  and  disadvantages  to  the  MDI  value  being  asset  specific? 
Advantages: 

Disadvantages: 

6)  What  are  the  advantages  and  disadvantages  to  the  MDI  value  be  dependent  on  groups 
such  as  CATCODEs? 

Advantages: 

Disadvantages: 

7)  Would  grouping  individual  assets  to  a  bundle  within  individual  assigned  CATCODEs  be 
appropriate? 

8)  What  is  the  desired  distribution  of  the  MDI  values? 

9)  At  what  level  should  the  MDI  values  be  consistent? 
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10)  Is  there  another  metric  the  CE  Career  field  should  use  rather  than  MDI  to  assist  in  the 
prioritization  of  projects? 

Appendix  E.  Round  I  Delphi  Coding  and  Combined  Answers 

1)  What  does  the  Mission  Dependency  Index  (MDI)  Value  represent  in  an  infrastructure 
portfolio? 


Theme 

Criticality 
AF  Mission 
Local  Mission 
Consequence  of 
Failure 
Importance 

2)  What  are  the  strengths  of  using  current  MDI  values? 

Theme 

Consistent 

Easy  to  use 

Objective 

Repeatable 

Common  frame  of 

reference 

Assists  in  prioritization 


3)  What  are  the  weaknesses  of  using  the  current  MDI  values? 

Theme 

Non-Flightline  missions  not  identified/not  objective 
CATCODEs/lack  of  individuality 
No  local  mission  impact 
No  account  of  redundancy 

Does  not  account  for  2nd/3rd/4th  order  effects/whose  mission? 

4)  What  inputs  need  to  be  taken  into  consideration  when  detennining  the  MDI  value  of  a 
facility?  Please  provide  at  least  three  inputs. 

Theme 

AF  Mission/DOD/COCOM 
Local  Mission/asset  specific 
Redundancy 
Replaceability 
Life/health/safety 
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Current  system 
Facility  type/usage 

5)  What  are  the  advantages  and  disadvantages  to  the  MDI  value  being  asset  specific? 


Advantages: 

Theme 

Relative  mission  impact 

Adaptability 

More  adaptable  to  non-standard  mission  sets 

Disadvantages: 

Theme 

Not  reflecting  the  AF  mission 

Inflation  of  values/non  mission  QOL  bldgs  getting  high  MDIs 
Expensive/time  and  money 

Do  not  know 

6)  What  are  the  advantages  and  disadvantages  to  the  MDI  value  be  dependent  on  groups 
such  as  CATCODEs? 


Advantages: 

Theme 

Easy/repeatable 

Standardized  across  the  AF/reputable 

Good  starting  point 

Disadvantages: 

Theme 

No  tie  to  local  mission/specific  Asset 

Bad  data 

7)  Would  grouping  individual  assets  to  a  bundle  within  individual  assigned  CATCODEs  be 
appropriate? 

Green  is  the  majority,  yellow  is  half,  red  is  the  minority 

Tjieme 

No 

Maybe 
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System  approach 


8)  What  is  the  desired  distribution  of  the  MDI  values? 

Theme 


Normal 

0-100 

50-100 


40-100 


Do  not  apply  one 
Did  not  understand 


9)  At  what  level  should  the  MDI  values  be  consistent? 


Theme 

AF 

MAJCOM 

BASE 

Mission 

none 

Did  not  understand 


10)  Is  there  another  metric  the  CE  Career  field  should  use  rather  than  MDI  to  assist  in  the 
prioritization  of  projects? 


Theme 

No 

PAL 

Local  Mission/asset  specific 
FUB 

CSAF  Priorities 
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Appendix  F.  Round  II  Delphi  Questions 


Mission  Dependency  Index  Metric: 

A  Value  Focused  Thinking  Study 
Primary  Researcher:  Captain  Matthew  Nichols 
United  States  Air  Force,  Air  Force  Institute  of  Technology 

Questions  Round  Two  of  the  Value  Focused  Thinking  Study 

1)  The  common  themes  identified  by  the  panel  for  the  Mission  Dependency  Index  (MDI) 
definition  are  shown  below.  Please  circle  or  highlight  whether  you  agree  or  disagree  with 
the  inputs. 

Criticality  to 


Local  Mission 

Agree  Disagree 

Air  Force  Mission 

Agree 

Disagree 

Consequence  of  Failure 
Measurement  of  mission  completion 

Agree 

Disagree 

using  that  specific  asset 

Agree 

Disagree 

Open  Response: 

2)  When  addressing  the  weaknesses  of  the  current  MDI,  the  panel  noted  that  current  MDI 
values  do  not  account  for  redundant  assets.  How  should  redundancy  be  accounted  for? 

Open  Response: 


3)  Another  weakness  identified  by  the  panel  in  Round  1  is  that  the  current  system  does 
account  for  the  replaceability  of  an  asset.  To  address  this  issue,  should  a  monetary, 
mission  impact,  or  another  metric  be  used  in  determining  the  MDI  value? 

Monetary  Mission  Impact  Other: _ 


Open  Response: 


4)  Currently,  the  MDI  is  applied  with  the  MAJCOM  Priority  to  determine  the  Consequence 
of  Failure  (CoF)  metric  for  the  SRM  funding  model.  (MDI  is  60%  and  the  MAJCOM 
Priority  is  40%) 


li. 


Does  the  MAJCOM  priority  adequately 
account  for  the  local  mission? 

Yes  No 


0  > 
ot 

o 

2* 
3  S 
~  O 

a  o 
U.  < 


Over  time,  should  the  weights  applied  to  MAJCOM  priority 
and  MDI  be  adjusted?  || 

Yes  No  gf 

0 

o 
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Crilktl  60 

SO 


Commander  Risk  Assessment 
Characterizes  Consequence 


1 _ t| 

Significant  Risk 


Moderate  Risk 


Engineering  Risk  Assessment 
Characterizes  Probability 


JL 


Probability  of  Failure  (PoF) 

Condition 


iii.  If  yes,  what  percentage  should  each  portion  be  to  detennine 
the  CoF  metric? 

MAJCOM  Priority _  MDI  Value _ 

Does  the  current  adjudication  process  lead  to  these  metrics  combining?  (please 
see  the  end  of  the  document  for  more  infonnation  on  the  adjudication  process) 

Open  Response: 

5)  Every  agency/organization  has  a  mission.  At  what  level  should  the  mission  effect  applied 
to  MDI  be  distinguished:  the  Air  Force,  MAJCOM,  or  Wing  level? 

6)  Should  each  mission  set  or  Core  Function  have  an  adjustment  factor  to  vary  the  MDI 
value  assigned  to  the  CATCODEs  associated  with  their  primary  mission?  (ex.  AETC’s 
classrooms  would  be  adjusted  to  be  higher  than  other  MAJCOMs,  please  see  the  end  of 
the  survey  for  the  Core  Functions  list) 

Yes  No 

Open  Response: 

7)  In  round  1 ,  the  panel  was  suggested  that  the  prioritization  happens  at  the  base  level 
through  the  use  of  the  Integrated  Priority  List  (IPL).  Should  there  be  a  portion  of  the  CoF 
that  originates  from  the  IPL? 

Yes  No 

i.  If  yes,  please  elaborate  on  how  the  MDI  metric  could  maintain  consistency  at  the 
prescribed  level  (AF,  MAJCOM  ,  or  Wing)  through  this  process. 

Open  Response: 


8)  One  team  member  stated,  “Since  the  installations  are  not  involved  in  the  process,  there  is 
no  buy-in  for  how  that  number  (MDI)  is  created.”  What  are  possible  courses  of  action  to 
fix  this? 

Open  Response: 

9)  One  team  member  pointed  out  that  the  MDI  is  used  for  funding  decisions.  Because  of 
this,  the  career  field  is  now  applying  an  asset  importance  metric  to  projects  (that  may 
impact  more  than  one  asset). 
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i.  When  this  is  the  case,  which  MDI  should  be  applied? 

Open  Response: 

ii.  Another  team  member  pointed  out  that  many  functions  are  often 
consolidated  into  one  asset.  Should  there  be  a  modification  to  the  current  MDI 
model  so  that  different  MDIs  could  be  applied  to  different  functional  areas  of  the 
asset? 

Open  Response: 


Core  Function  Lead  Integrators  (in  green)  &  OCRs  (indicated  by  checkmarks) 


ACC 

AETC 

AFGSC 

AFMC 

AFRC 

AFSOC 

AFSPC 

AMC 

NGB 

PACAF 

USAFE 

HAF 

Nuclear  Deterrence  Ops 

✓ 

✓ 

AFGSC/ 

cc 

/ 

/ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

A10 

Air  Superiority 

ACC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

A3/5 

Space  Superiority 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

AFSPC/ 

CC 

✓ 

✓ 

✓ 

✓ 

A3/5 

Cyberspace  Superiority 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

AFSPC/ 

CC 

✓ 

✓ 

✓ 

✓ 

A3/5,  A6 

Global  Precision  Attack 

ACC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

A3/5 

Rapid  Global  Mobility 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

AM  C/C  C 

✓ 

✓ 

✓ 

A3/5 

Special  Operations 

✓ 

✓ 

✓ 

✓ 

✓ 

AFSOC/  CC 

✓ 

✓ 

✓ 

✓ 

✓ 

A3/5 

Global  Integrated  ISR 

ACC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

A2 

Command  and  Control 

ACC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

A2,  A3/5, 
A4/7,  A6 

Personnel  Recovery 

ACC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

A3/5 

Building  Partnerships 

✓ 

AETC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

IA 

Agile  Combat  Support 

✓ 

✓ 

✓ 

AFMC/CC 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

AQ.  A1, 

A3/5,  A4/7 

The  current  adjudication  process  allows  for  MAJCOMs  to  suggest  a  change  in  the  MDI  value 
assigned  to  specific  CATCODEs.  The  entire  process  is  outlined  in  the  CoF  playbook  below. 
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Appendix  G.  Round  II  Delphi  Coding  and  Combined  Answers 

1)  The  common  themes  identified  by  the  panel  for  the  Mission  Dependency  Index  (MDI) 
definition  are  shown  below. 


Local  Mission 

Agree:  5 

Disagree:  1 

Air  Force  Mission 

Agree:  5 

Disagreed 

Consequence  of  Failure 

Agree:  6 

Disagreed 

Measurement  of  mission 
completion  using  that 
specific  asset 

Agree:  3 

Disagree:! 

Neither:  2 

2)  When  addressing  the  weaknesses  of  the  current  MDI,  the  panel  noted  that  current  MDI 
values  do  not  account  for  redundant  assets.  How  should  redundancy  be  accounted  for? 

Theme 

Money 

Time 

Local  Redundancy 
AF  Wide  Redundancy 
With  a  CFLI  Coefficient 
Mission  Capacity 


3)  Another  weakness  identified  by  the  panel  in  Round  1  is  that  the  current  system  does 
account  for  the  replaceability  of  an  asset.  To  address  this  issue,  should  a  monetary, 
mission  impact,  or  another  metric  be  used  in  determining  the  MDI  value? 


Theme 

Money 

Mission 

Impact 

Time 


4)  Currently,  the  MDI  is  applied  with  the  MAJCOM  Priority  to  determine  the  Consequence 
of  Failure  (CoF)  metric  for  the  SRM  funding  model.  (MDI  is  60%  and  the  MAJCOM 
Priority  is  40%) 

a.  Does  the  MAJCOM  priority  adequately  account  for  the  local  mission? 

Yes  i\o 

b.  Over  time,  should  the  weights  applied  to  MAJCOM  priority  and  MDI  be 
adjusted? 

Yes  No 
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c.  If  yes,  what  percentage  should  each  portion  be  to  determine 
the  CoF  metric?  Two  groups 

MAJCOM  MDI  Value 
Priority 

TBD  TBD 

0  100 

Does  the  current  adjudication  process  lead  to  these  metrics  combining?  Yes 
Theme 

Local 

MAJCOM 

HAF 

Commander  Influence 


5)  Every  agency/organization  has  a  mission.  At  what  level  should  the  mission  effect  applied 
to  MDI  be  distinguished:  the  Air  Force,  MAJCOM,  or  Wing  level? 

Two  Groups 

Theme 

AF 

Wing 


6)  Should  each  mission  set  or  Core  Function  have  an  adjustment  factor  to  vary  the  MDI 
value  assigned  to  the  CATCODEs  associated  with  their  primary  mission?  (ex.  AETC’s 
classrooms  would  be  adjusted  to  be  higher  than  other  MAJCOMs,  please  see  the  end  of 
the  survey  for  the  Core  Functions  list) 

Yes  No 

7)  In  round  1 ,  the  panel  was  suggested  that  the  prioritization  happens  at  the  base  level 
through  the  use  of  the  Integrated  Priority  List  (IPL).  Should  there  be  a  portion  of  the  CoF 
that  originates  from  the  IPL? 

Yes  No 


a.  If  yes,  please  elaborate  on  how  the  MDI  metric  could  maintain  consistency  at  the 
prescribed  level  (AF,  MAJCOM  ,  or  Wing)  through  this  process. 
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N/A 


8)  One  team  member  stated,  “Since  the  installations  are  not  involved  in  the  process,  there  is 
no  buy-in  for  how  that  number  (MDI)  is  created.”  What  are  possible  courses  of  action  to 
fix  this? 

Theme 

Adjudication  process 

Involve  local  leadership  in  development  of 
value 

Have  each  MAJCOM  articulate  issues 


9)  One  team  member  pointed  out  that  the  MDI  is  used  for  funding  decisions.  Because  of 
this,  the  career  field  is  now  applying  an  asset  importance  metric  to  projects  (that  may 
impact  more  than  one  asset). 

a.  When  this  is  the  case,  which  MDI  should  be  applied? 

b.  Another  team  member  pointed  out  that  many  functions  are  often  consolidated  into 
one  asset.  Should  there  be  a  modification  to  the  current  MDI  model  so  that 
different  MDIs  could  be  applied  to  different  functional  areas  of  the  asset? 

Evenly  split  throughout  themes 

Theme 

Weighted 

average 

Highest  Value 

Requirement 

specific 
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Appendix  H.  Round  III  Delphi  Questions 


Mission  Dependency  Index  Metric: 

A  Value  Focused  Thinking  Study 
Primary  Researcher:  Captain  Matthew  Nichols 
United  States  Air  Force,  Air  Force  Institute  of  Technology 

Questions  Round  Three  of  the  Value  Focused  Thinking  Study 

1)  Throughout  the  last  two  rounds,  the  panel  has  identified  the  following  inputs  that  MDI 
should  address;  please  place  the  weights  you  believe  each  section  should  have.  All  of  the 
weights  should  sum  to  equal  100. 

Criticality  to  the  local  mission  (Wing):  _ 

Criticality  to  the  AF/MAJCOM  Mission  (Core  Function):  _ 

Redundancy:  _ 

Time  to  replace:  _ 

Cost:  _ 

Number  of  local  missions  impacted:  _ 

Number  of  Core  Functions  impacted:  _ 

Open  Response: 

2)  One  member  stated,  “We  are  trying  to  make  a  system  which  circumvents  commander 
influence;  I  believe  their  job  is  to  influence.”  Other  members  of  the  panel  believed  the 
Consequence  of  Failure  (CoF)  metric  should  have  the  MAJCOM  priority  removed  with 
just  the  MDI  remaining.  After  accomplishing  this  task,  the  MAJCOM  priority  points 
should  be  applied  separately  from  the  CoF  and  Probability  of  Failure  (PoF)  graph.  This 
method  may  be  able  to  reduce  the  fair  share  and  “gaming”  the  system  that  currently 
happens. 

Do  you  agree  with  this  methodology  or  is  there  another  possible  route  to  take? 

Open  Response: 

3)  Another  member  brought  up,  “Should  services  that  are  available  in  the  surrounding 
community  be  taken  into  consideration  when  looking  at  the  redundancy  of  an  asset?”  If 
the  service  provided  by  the  facility  can  be  absorbed  by  the  community  in  the  event  of 
asset  failure  should  this  be  addressed  when  establishing  the  MDI  value? 

Yes:  No: 

Open  Response: 

4)  One  panel  member  pointed  out,  the  Navy’s  methodology  works  for  them  and  addresses 
most  of  the  issues  the  Air  Force’s  process  has. 
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NAVFAC  sends  a  team  out  that  asks  local  commanders  about  local  impacts  as  well  and 
the  impacts  of  others  for  each  asset  and  then  nonnalizes  the  MDI  values  using  an 
equation  that  is  84%  intra-dependency,  1 1%  inter-dependency  and  5%  the  number  of 
leaders  that  believe  the  asset  is  important,  (full  methodology  available  in  the  PDF  at  the 
end  of  the  document) 

Would  you  as  an  expert,  suggest  for  the  Air  Force  to  fund  a  team  to  establish  MDI  values 
using  the  Navy  methodology? 

Yes:  No: 

Open  Response: 

5)  Any  suggestions  for  future  research  areas  for  AFIT  of  AFCEC? 
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Appendix  I.  Round  III  Delphi  Coding  and  Combined  Answers 

1)  Throughout  the  last  two  rounds,  the  panel  has  identified  the  following  inputs  that  MDI 
should  address;  please  place  the  weights  you  believe  each  section  should  have.  All  of  the 
weights  should  sum  to  equal  100. 


Panel  Member 

Values  for  the  MDI  Metric 

A 

B 

c 

D 

E 

Criticality  at  the  Local  level 

30 

40 

50 

90 

25 

Criticality  of  the  Core  Functions  (AF 
level) 

20 

40 

10 

5 

35 

Redundancy 

35 

10 

20 

5 

15 

Time  to  replace 

10 

10 

0 

0 

0 

Cost 

0 

0 

0 

0 

0 

Number  of  Local  Level  Missions 
Impacted 

0 

0 

10 

0 

10 

Number  of  Core  Functions  Missions 
Impacted  (AF  level) 

0 

0 

10 

0 

15 

2)  One  member  stated,  “We  are  trying  to  make  a  system  which  circumvents  commander 
influence;  I  believe  their  job  is  to  influence.”  Other  members  of  the  panel  believed  the 
Consequence  of  Failure  (CoF)  metric  should  have  the  MAJCOM  priority  removed  with 
just  the  MDI  remaining.  After  accomplishing  this  task,  the  MAJCOM  priority  points 
should  be  applied  separately  from  the  CoF  and  Probability  of  Failure  (PoF)  graph.  This 
method  may  be  able  to  reduce  the  fair  share  and  “gaming”  the  system  that  currently 
happens.  Do  you  agree  with  this  methodology  or  is  there  another  possible  route  to  take? 

2  groups 

Theme 

Minimize  local  Commander's 
influence 

Maximize  local  Commander's 
influence 

3)  Another  member  brought  up,  “Should  services  that  are  available  in  the  surrounding 
community  be  taken  into  consideration  when  looking  at  the  redundancy  of  an  asset?”  If 
the  service  provided  by  the  facility  can  be  absorbed  by  the  community  in  the  event  of 
asset  failure  should  this  be  addressed  when  establishing  the  MDI  value? 

Yes:  Half  No  :  Half 

4)  One  panel  member  pointed  out,  the  Navy’s  methodology  works  for  them  and  addresses 
most  of  the  issues  the  Air  Force’s  process  has. 

NAVFAC  sends  a  team  out  that  asks  local  commanders  about  local  impacts  as  well  and 
the  impacts  of  others  for  each  asset  and  then  nonnalizes  the  MDI  values  using  an 
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equation  that  is  84%  intra-dependency,  11%  inter-dependency  and  5%  the  number  of 
leaders  that  believe  the  asset  is  important,  (full  methodology  available  in  the  PDF  at  the 
end  of  the  document) 

Would  you  as  an  expert,  suggest  for  the  Air  Force  to  fund  a  team  to  establish  MDI  values 
using  the  Navy  methodology? 

Yes:  Half  No:  Half 
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Appendix  J.  AFCEC  Sponsor  Letter  to  Delphi  Participants 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  EDUCATION  AND  TRAINING  COMMAND 


6  October  2014 


MEMORANDUM  FOR  VALUE  FOCUSED  THINKING  STUDY  MEMBERS 
FROM:  Lieutenant  Colonel  Chad  B.  BonDurant,  AFCEC/CPAD 

SUBJECT:  Significance  of  Value  Focused  Thinking  (VFT)  Study  on  Mission  Dependency 
Index  (MDI)  Values 

1 .  the  Air  Force  Civil  Engineer  Center  (AFCEC)  is  sponsoring  an  Air  Force  Institute  of 
Technology  (AFIT)  thesis  effort  to  better  define  the  Mission  Dependency  Index  (MDI)  value 
distribution,  definition,  and  adaptability,  the  MDI  is  generated  automatically  based  on 
CATCODE.  The  current  business  rules  allow  MAJCOMs  to  submit  an  MDI  adjustment  request 
for  specific  facilities  if  unique  circumstances  exist.  However,  AFCEC  is  concerned  this  may 
cause  MDI  values  to  inflate  over  time  reducing  their  decision  making  value.  This  research  is  an 
effort  to  help  identify  a  systematic  and  transparent  approach  to  keep  MDI  values  in  the  desired 
distribution. 

2.  This  VFT  study  seeks  to  determine  what  value  senior  asset  managers  and  leaders  believe  the 
MDI  metric  provides.  The  MDI  values  are  currently  assessed  from  an  Alternative-Focused 
Thinking  (AF  I  )  mindset.  The  AFT  framework  looks  at  all  identified  alternatives  and  ranks  the 
options  from  that  point  of  view.  However,  one  major  flaw  with  the  AFT  style  of  decision¬ 
making  is  that  if  the  alternative  is  not  identified,  it  cannot  compete,  or  one  alternative  is 
overemphasized. 

3.  The  VFT  approach  identifies  the  overarching  goals  of  a  metric  and  uses  those  metrics  to 
establish  a  model.  This  approach  is  beneficial  because  it  creates  a  repeatable  process  that  can  be 
applied  to  any  alternative,  even  those  identified  later.  Any  alternatives  evaluated  with  this  model 
will  then  be  consistent  with  the  corporate  goals  identified  through  this  research  effort. 

4.  The  current  MDI  was  produced  by  adapting  NAVFAC  facility  data  for  use  with  USAF 
facilities.  This  research  stream  is  not  designed  to  overhaul  the  current  MDI  system.  It  is 
designed  to  provide  a  more  defendable  scoring  model  for  the  centralized  program.  This  research 
in  combination  with  the  recently  developed  playbook  will  enhance  the  USAF  MDI  values 
reliability. 

5.  If  you  have  any  questions  or  concerns  about  this  study  please  feel  free  to  contact  the  action 
officer.  Captain  Matt  Nichols  at  303-895-5580  or  matthcw  .nichols'V/jafit.edu, 


CHAD  B.  BONDURANT,  Lt  Col,  USAF 

Chief,  Comprehensive  Program  Development  Branch 

Air  Force  Civil  Engineer  Center 
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Appendix  K.  AFIT  Human  Subjects  Exemption  Approval 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 
WRIGHT-PATTERSON  AIR  FORCE  BASE  OHIO 


22  Sept  2014 


MEMORANDUM  FOR  DR.  AL  THAL 

FROM:  Jeffrey  A.  Ogden,  PhD. 

AFTT  IRB  Research  Reviewer 
2950  Hobson  Way 

Wnght-Patterson  AFB,  OH  45433-7765 

SUBJECT:  Approval  for  exemption  request  from  human  expemnentation  requirements  (32  CFR 
219,  DoDD  3216.2  and  AFT  40-402)  for  the  Mission  Dependency  Index  Value  Focused  Thmking 
Model 

1.  Your  request  was  based  on  the  Code  of  Federal  Regulations,  title  32,  part  219,  section  101, 
paragraph  (b)  (2)  Research  activities  that  involve  the  use  of  educational  tests  (cognitive, 
diagnostic,  aptitude,  achievement),  survey  procedures,  interview'  procedures,  or  observation  of 
pubhc  behavior  unless:  (i)  Information  obtained  is  recorded  m  such  a  manner  that  human 
subjects  can  be  identified,  directly  or  through  identifiers  lmked  to  the  subjects;  and  (u)  Any 
disclosure  of  the  human  subjects’  responses  outside  the  research  could  reasonably  place  the 
subjects  at  nsk  of  criminal  or  civil  liability  or  be  damaging  to  the  subjects’  financial  standing, 
employability,  or  reputation. 

2.  Your  study  qualifies  for  this  exemption  because  you  are  not  collecting  and  reporting  sensitive 
data,  w'hich  could  reasonably  damage  the  subjects’  financial  standing,  employability,  or 
reputation.  Further,  you  are  not  collecting  and  reporting  any  demographic  data  w  hich  could 
realistically  be  expected  to  map  a  given  response  to  a  specific  subject. 

3.  This  determination  pertains  only  to  the  Federal,  Department  of  Defense,  and  Air  Force 
regulations  that  govern  the  use  of  human  subjects  in  research.  Further,  if  a  subject’s  future 
response  reasonably  places  them  at  risk  of  criminal  or  civil  liability  or  is  damaging  to  their 
financial  standing,  employability,  or  reputation,  you  are  required  to  file  an  adverse  event  report 
with  this  office  immediately. 


9/22/2014 


X  Jeffrey  A.  Ogden 

Jeffrey  A.  Ogden,  Ph.D. 

IRB  Exempt  Determination  Official 
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